Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/schnet_feat_avg_just_mpnn_from_binary_cross_entropy/config.json --data_path /home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_avg_just_mpnn_from_binary_cross_entropy/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv',
 'dataset_type': 'classification',
 'depth': 3,
 'device': device(type='cuda', index=1),
 'dropout': 0.0,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': None,
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 1700,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 1700,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'binary_cross_entropy',
 'metrics': ['binary_cross_entropy'],
 'minimize_score': True,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/schnet_feat_avg_just_mpnn_from_binary_cross_entropy',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': None,
 'separate_test_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/test_full.csv',
 'separate_val_features_path': None,
 'separate_val_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': False,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1700, bias=False)
        (W_h): Linear(in_features=1700, out_features=1700, bias=False)
        (W_o): Linear(in_features=1833, out_features=1700, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=1700, out_features=1700, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=1700, out_features=1700, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.0, inplace=False)
    (7): Linear(in_features=1700, out_features=1, bias=True)
  )
)
Number of parameters = 12,042,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.575757
Epoch 1
Validation binary_cross_entropy = 0.654255
Epoch 2
Validation binary_cross_entropy = 0.690855
Epoch 3
Validation binary_cross_entropy = 0.753638
Epoch 4
Loss = 6.3625e-01, PNorm = 84.5247, GNorm = 0.5899, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.539326
Epoch 5
Validation binary_cross_entropy = 0.368219
Epoch 6
Validation binary_cross_entropy = 0.732139
Epoch 7
Validation binary_cross_entropy = 0.582165
Epoch 8
Validation binary_cross_entropy = 0.481271
Epoch 9
Loss = 5.2178e-01, PNorm = 84.8044, GNorm = 1.0060, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 1.119090
Epoch 10
Validation binary_cross_entropy = 0.305154
Epoch 11
Validation binary_cross_entropy = 0.585268
Epoch 12
Validation binary_cross_entropy = 0.603198
Epoch 13
Validation binary_cross_entropy = 0.574730
Epoch 14
Loss = 5.2172e-01, PNorm = 85.1760, GNorm = 0.7383, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.366803
Epoch 15
Validation binary_cross_entropy = 0.512730
Epoch 16
Validation binary_cross_entropy = 0.575258
Epoch 17
Validation binary_cross_entropy = 0.301491
Epoch 18
Validation binary_cross_entropy = 0.656576
Epoch 19
Loss = 4.4766e-01, PNorm = 85.5872, GNorm = 1.4249, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.355281
Epoch 20
Validation binary_cross_entropy = 0.327709
Epoch 21
Validation binary_cross_entropy = 0.399546
Epoch 22
Validation binary_cross_entropy = 0.342582
Epoch 23
Validation binary_cross_entropy = 0.455377
Epoch 24
Loss = 3.8803e-01, PNorm = 85.9756, GNorm = 1.3350, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.481452
Epoch 25
Validation binary_cross_entropy = 0.453721
Epoch 26
Validation binary_cross_entropy = 0.327084
Epoch 27
Validation binary_cross_entropy = 0.377180
Epoch 28
Validation binary_cross_entropy = 0.319029
Epoch 29
Loss = 4.4310e-01, PNorm = 86.3667, GNorm = 3.4529, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.540396
Epoch 30
Validation binary_cross_entropy = 0.621334
Epoch 31
Validation binary_cross_entropy = 0.319510
Epoch 32
Validation binary_cross_entropy = 0.362549
Epoch 33
Validation binary_cross_entropy = 0.470807
Epoch 34
Loss = 3.2833e-01, PNorm = 86.7621, GNorm = 0.5252, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.364633
Epoch 35
Validation binary_cross_entropy = 0.340210
Epoch 36
Validation binary_cross_entropy = 0.408197
Epoch 37
Validation binary_cross_entropy = 0.514777
Epoch 38
Validation binary_cross_entropy = 0.309276
Epoch 39
Loss = 3.0309e-01, PNorm = 87.1355, GNorm = 0.3980, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.350312
Epoch 40
Validation binary_cross_entropy = 0.403672
Epoch 41
Validation binary_cross_entropy = 0.436649
Epoch 42
Validation binary_cross_entropy = 0.518574
Epoch 43
Validation binary_cross_entropy = 0.947211
Epoch 44
Loss = 6.9879e-01, PNorm = 87.5117, GNorm = 3.9685, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 1.058157
Epoch 45
Validation binary_cross_entropy = 0.344968
Epoch 46
Validation binary_cross_entropy = 0.256286
Epoch 47
Validation binary_cross_entropy = 0.299083
Epoch 48
Validation binary_cross_entropy = 0.386451
Epoch 49
Loss = 4.2965e-01, PNorm = 87.9430, GNorm = 0.4865, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.402906
Epoch 50
Validation binary_cross_entropy = 0.350972
Epoch 51
Validation binary_cross_entropy = 0.322157
Epoch 52
Validation binary_cross_entropy = 0.399880
Epoch 53
Validation binary_cross_entropy = 0.448502
Epoch 54
Loss = 3.3748e-01, PNorm = 88.4438, GNorm = 0.7345, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.423558
Epoch 55
Validation binary_cross_entropy = 0.486968
Epoch 56
Validation binary_cross_entropy = 0.529902
Epoch 57
Validation binary_cross_entropy = 0.550104
Epoch 58
Validation binary_cross_entropy = 0.540691
Epoch 59
Loss = 1.9114e-01, PNorm = 88.8513, GNorm = 0.8511, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.557139
Epoch 60
Validation binary_cross_entropy = 0.610453
Epoch 61
Validation binary_cross_entropy = 0.690537
Epoch 62
Validation binary_cross_entropy = 0.711610
Epoch 63
Validation binary_cross_entropy = 0.703631
Epoch 64
Loss = 1.2094e-01, PNorm = 89.2155, GNorm = 1.6611, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.718137
Epoch 65
Validation binary_cross_entropy = 0.677121
Epoch 66
Validation binary_cross_entropy = 0.593212
Epoch 67
Validation binary_cross_entropy = 0.681054
Epoch 68
Validation binary_cross_entropy = 0.689632
Epoch 69
Loss = 1.3667e-01, PNorm = 89.5261, GNorm = 0.6968, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.650150
Epoch 70
Validation binary_cross_entropy = 0.677299
Epoch 71
Validation binary_cross_entropy = 0.690765
Epoch 72
Validation binary_cross_entropy = 0.700538
Epoch 73
Validation binary_cross_entropy = 0.762106
Epoch 74
Loss = 7.1685e-02, PNorm = 89.8270, GNorm = 0.6561, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.810502
Epoch 75
Validation binary_cross_entropy = 0.878699
Epoch 76
Validation binary_cross_entropy = 0.888752
Epoch 77
Validation binary_cross_entropy = 0.901522
Epoch 78
Validation binary_cross_entropy = 0.942141
Epoch 79
Loss = 1.3908e-01, PNorm = 90.0804, GNorm = 1.4087, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.942428
Epoch 80
Validation binary_cross_entropy = 0.856441
Epoch 81
Validation binary_cross_entropy = 0.831526
Epoch 82
Validation binary_cross_entropy = 0.847565
Epoch 83
Validation binary_cross_entropy = 0.943597
Epoch 84
Loss = 9.6536e-02, PNorm = 90.2836, GNorm = 0.6298, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.881030
Epoch 85
Validation binary_cross_entropy = 0.889170
Epoch 86
Validation binary_cross_entropy = 0.883574
Epoch 87
Validation binary_cross_entropy = 0.902292
Epoch 88
Validation binary_cross_entropy = 0.931324
Epoch 89
Loss = 1.0732e-01, PNorm = 90.4917, GNorm = 0.2528, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.927726
Epoch 90
Validation binary_cross_entropy = 0.970786
Epoch 91
Validation binary_cross_entropy = 0.933880
Epoch 92
Validation binary_cross_entropy = 0.944484
Epoch 93
Validation binary_cross_entropy = 1.022307
Epoch 94
Loss = 8.2986e-02, PNorm = 90.6590, GNorm = 2.6119, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.022673
Epoch 95
Validation binary_cross_entropy = 1.034571
Epoch 96
Validation binary_cross_entropy = 1.061788
Epoch 97
Validation binary_cross_entropy = 0.947312
Epoch 98
Validation binary_cross_entropy = 0.862041
Epoch 99
Loss = 4.5395e-02, PNorm = 90.7990, GNorm = 0.2856, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.871748
Epoch 100
Validation binary_cross_entropy = 0.916861
Epoch 101
Validation binary_cross_entropy = 0.981656
Epoch 102
Validation binary_cross_entropy = 0.984410
Epoch 103
Validation binary_cross_entropy = 0.909789
Epoch 104
Loss = 6.7699e-02, PNorm = 90.9621, GNorm = 0.4960, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.896617
Epoch 105
Validation binary_cross_entropy = 0.896668
Epoch 106
Validation binary_cross_entropy = 0.880258
Epoch 107
Validation binary_cross_entropy = 0.925135
Epoch 108
Validation binary_cross_entropy = 0.979190
Epoch 109
Loss = 6.6699e-02, PNorm = 91.1186, GNorm = 0.5414, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 1.106838
Epoch 110
Validation binary_cross_entropy = 1.016744
Epoch 111
Validation binary_cross_entropy = 1.003996
Epoch 112
Validation binary_cross_entropy = 1.118220
Epoch 113
Validation binary_cross_entropy = 1.076665
Epoch 114
Loss = 6.0040e-02, PNorm = 91.2738, GNorm = 1.5676, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.968126
Epoch 115
Validation binary_cross_entropy = 0.918523
Epoch 116
Validation binary_cross_entropy = 0.953179
Epoch 117
Validation binary_cross_entropy = 0.919554
Epoch 118
Validation binary_cross_entropy = 0.916697
Epoch 119
Loss = 7.2132e-02, PNorm = 91.4601, GNorm = 1.6546, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 1.079287
Epoch 120
Validation binary_cross_entropy = 1.112295
Epoch 121
Validation binary_cross_entropy = 1.134264
Epoch 122
Validation binary_cross_entropy = 1.197410
Epoch 123
Validation binary_cross_entropy = 1.348724
Epoch 124
Loss = 1.1713e-01, PNorm = 91.6585, GNorm = 3.2365, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 1.396304
Epoch 125
Validation binary_cross_entropy = 1.341713
Epoch 126
Validation binary_cross_entropy = 1.248856
Epoch 127
Validation binary_cross_entropy = 1.092501
Epoch 128
Validation binary_cross_entropy = 0.933278
Epoch 129
Loss = 2.9471e-02, PNorm = 91.8434, GNorm = 0.1382, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.871567
Epoch 130
Validation binary_cross_entropy = 0.915290
Epoch 131
Validation binary_cross_entropy = 0.917153
Epoch 132
Validation binary_cross_entropy = 0.960821
Epoch 133
Validation binary_cross_entropy = 1.015765
Epoch 134
Loss = 4.1276e-02, PNorm = 92.0580, GNorm = 1.0244, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 1.106846
Epoch 135
Validation binary_cross_entropy = 1.196040
Epoch 136
Validation binary_cross_entropy = 1.243420
Epoch 137
Validation binary_cross_entropy = 1.283818
Epoch 138
Validation binary_cross_entropy = 1.350762
Epoch 139
Loss = 4.1338e-02, PNorm = 92.2514, GNorm = 0.0952, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 1.404024
Epoch 140
Validation binary_cross_entropy = 1.441552
Epoch 141
Validation binary_cross_entropy = 1.464760
Epoch 142
Validation binary_cross_entropy = 1.435347
Epoch 143
Validation binary_cross_entropy = 1.425315
Epoch 144
Loss = 7.7984e-03, PNorm = 92.3871, GNorm = 0.1697, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.430426
Epoch 145
Validation binary_cross_entropy = 1.579980
Epoch 146
Validation binary_cross_entropy = 1.758872
Epoch 147
Validation binary_cross_entropy = 1.877613
Epoch 148
Validation binary_cross_entropy = 1.797636
Epoch 149
Loss = 1.5238e-02, PNorm = 92.4826, GNorm = 0.4583, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.724197
Epoch 150
Validation binary_cross_entropy = 1.669212
Epoch 151
Validation binary_cross_entropy = 1.645543
Epoch 152
Validation binary_cross_entropy = 1.634466
Epoch 153
Validation binary_cross_entropy = 1.652203
Epoch 154
Loss = 1.3861e-02, PNorm = 92.5563, GNorm = 0.6085, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 1.648043
Epoch 155
Validation binary_cross_entropy = 1.537531
Epoch 156
Validation binary_cross_entropy = 1.508692
Epoch 157
Validation binary_cross_entropy = 1.530858
Epoch 158
Validation binary_cross_entropy = 1.544877
Epoch 159
Loss = 8.5165e-03, PNorm = 92.6281, GNorm = 0.1791, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.562188
Epoch 160
Validation binary_cross_entropy = 1.575706
Epoch 161
Validation binary_cross_entropy = 1.583415
Epoch 162
Validation binary_cross_entropy = 1.629348
Epoch 163
Validation binary_cross_entropy = 1.733189
Epoch 164
Loss = 9.7561e-02, PNorm = 92.7123, GNorm = 4.3897, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.678131
Epoch 165
Validation binary_cross_entropy = 1.493518
Epoch 166
Validation binary_cross_entropy = 1.388431
Epoch 167
Validation binary_cross_entropy = 1.332545
Epoch 168
Validation binary_cross_entropy = 1.321965
Epoch 169
Loss = 5.3014e-03, PNorm = 92.8362, GNorm = 0.0829, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.316380
Epoch 170
Validation binary_cross_entropy = 1.335826
Epoch 171
Validation binary_cross_entropy = 1.351057
Epoch 172
Validation binary_cross_entropy = 1.363151
Epoch 173
Validation binary_cross_entropy = 1.350040
Epoch 174
Loss = 2.6674e-02, PNorm = 92.9439, GNorm = 1.4248, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.395677
Epoch 175
Validation binary_cross_entropy = 1.476259
Epoch 176
Validation binary_cross_entropy = 1.538731
Epoch 177
Validation binary_cross_entropy = 1.564062
Epoch 178
Validation binary_cross_entropy = 1.582079
Epoch 179
Loss = 1.7632e-03, PNorm = 93.0332, GNorm = 0.0445, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.602582
Epoch 180
Validation binary_cross_entropy = 1.620360
Epoch 181
Validation binary_cross_entropy = 1.637168
Epoch 182
Validation binary_cross_entropy = 1.642583
Epoch 183
Validation binary_cross_entropy = 1.697034
Epoch 184
Loss = 5.7475e-03, PNorm = 93.0999, GNorm = 0.3468, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.765214
Epoch 185
Validation binary_cross_entropy = 1.754701
Epoch 186
Validation binary_cross_entropy = 1.715632
Epoch 187
Validation binary_cross_entropy = 1.693181
Epoch 188
Validation binary_cross_entropy = 1.690014
Epoch 189
Loss = 4.3680e-03, PNorm = 93.1524, GNorm = 0.1191, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.702881
Epoch 190
Validation binary_cross_entropy = 1.686493
Epoch 191
Validation binary_cross_entropy = 1.717221
Epoch 192
Validation binary_cross_entropy = 1.787836
Epoch 193
Validation binary_cross_entropy = 1.671404
Epoch 194
Loss = 2.1989e-03, PNorm = 93.2236, GNorm = 0.0228, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.613341
Epoch 195
Validation binary_cross_entropy = 1.597270
Epoch 196
Validation binary_cross_entropy = 1.669919
Epoch 197
Validation binary_cross_entropy = 1.806188
Epoch 198
Validation binary_cross_entropy = 1.935426
Epoch 199
Loss = 3.1169e-03, PNorm = 93.2942, GNorm = 0.1328, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 2.024907
Epoch 200
Validation binary_cross_entropy = 2.017267
Epoch 201
Validation binary_cross_entropy = 2.006597
Epoch 202
Validation binary_cross_entropy = 2.000414
Epoch 203
Validation binary_cross_entropy = 1.996010
Epoch 204
Loss = 2.2644e-04, PNorm = 93.3384, GNorm = 0.0056, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.992130
Epoch 205
Validation binary_cross_entropy = 1.988639
Epoch 206
Validation binary_cross_entropy = 1.983912
Epoch 207
Validation binary_cross_entropy = 1.983639
Epoch 208
Validation binary_cross_entropy = 1.993504
Epoch 209
Loss = 4.7886e-04, PNorm = 93.3714, GNorm = 0.0046, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 2.006015
Epoch 210
Validation binary_cross_entropy = 2.017128
Epoch 211
Validation binary_cross_entropy = 2.027619
Epoch 212
Validation binary_cross_entropy = 2.036583
Epoch 213
Validation binary_cross_entropy = 2.043212
Epoch 214
Loss = 4.7168e-04, PNorm = 93.3943, GNorm = 0.0152, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 2.048423
Epoch 215
Validation binary_cross_entropy = 2.051995
Epoch 216
Validation binary_cross_entropy = 2.054937
Epoch 217
Validation binary_cross_entropy = 2.058192
Epoch 218
Validation binary_cross_entropy = 2.061812
Epoch 219
Loss = 2.7157e-04, PNorm = 93.4074, GNorm = 0.0074, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 2.065205
Epoch 220
Validation binary_cross_entropy = 2.068378
Epoch 221
Validation binary_cross_entropy = 2.071126
Epoch 222
Validation binary_cross_entropy = 2.073833
Epoch 223
Validation binary_cross_entropy = 2.076640
Epoch 224
Loss = 4.7214e-04, PNorm = 93.4162, GNorm = 0.0033, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 2.080654
Epoch 225
Validation binary_cross_entropy = 2.159919
Epoch 226
Validation binary_cross_entropy = 2.324046
Epoch 227
Validation binary_cross_entropy = 2.366200
Epoch 228
Validation binary_cross_entropy = 2.305030
Epoch 229
Loss = 3.7915e-04, PNorm = 93.4427, GNorm = 0.0078, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 2.255570
Epoch 230
Validation binary_cross_entropy = 2.216928
Epoch 231
Validation binary_cross_entropy = 2.186843
Epoch 232
Validation binary_cross_entropy = 2.163182
Epoch 233
Validation binary_cross_entropy = 2.145387
Epoch 234
Loss = 2.0992e-04, PNorm = 93.4603, GNorm = 0.0014, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 2.133088
Epoch 235
Validation binary_cross_entropy = 2.123008
Epoch 236
Validation binary_cross_entropy = 2.115213
Epoch 237
Validation binary_cross_entropy = 2.110671
Epoch 238
Validation binary_cross_entropy = 2.107430
Epoch 239
Loss = 7.9600e-05, PNorm = 93.4702, GNorm = 0.0011, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 2.104558
Epoch 240
Validation binary_cross_entropy = 2.104569
Epoch 241
Validation binary_cross_entropy = 2.107093
Epoch 242
Validation binary_cross_entropy = 2.111743
Epoch 243
Validation binary_cross_entropy = 2.119929
Epoch 244
Loss = 1.5598e-04, PNorm = 93.4798, GNorm = 0.0034, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 2.126758
Epoch 245
Validation binary_cross_entropy = 2.131564
Epoch 246
Validation binary_cross_entropy = 2.134896
Epoch 247
Validation binary_cross_entropy = 2.138433
Epoch 248
Validation binary_cross_entropy = 2.140556
Epoch 249
Loss = 2.0532e-04, PNorm = 93.4873, GNorm = 0.0032, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 2.142579
Epoch 250
Validation binary_cross_entropy = 2.143852
Epoch 251
Validation binary_cross_entropy = 2.144949
Epoch 252
Validation binary_cross_entropy = 2.145346
Epoch 253
Validation binary_cross_entropy = 2.145645
Epoch 254
Loss = 1.5424e-04, PNorm = 93.4924, GNorm = 0.0044, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 2.146149
Epoch 255
Validation binary_cross_entropy = 2.146832
Epoch 256
Validation binary_cross_entropy = 2.147630
Epoch 257
Validation binary_cross_entropy = 2.148152
Epoch 258
Validation binary_cross_entropy = 2.148731
Epoch 259
Loss = 1.2739e-04, PNorm = 93.4963, GNorm = 0.0027, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 2.149870
Epoch 260
Validation binary_cross_entropy = 2.151343
Epoch 261
Validation binary_cross_entropy = 2.153303
Epoch 262
Validation binary_cross_entropy = 2.155234
Epoch 263
Validation binary_cross_entropy = 2.156771
Epoch 264
Loss = 1.5586e-04, PNorm = 93.5000, GNorm = 0.0074, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 2.159281
Epoch 265
Validation binary_cross_entropy = 2.162136
Epoch 266
Validation binary_cross_entropy = 2.164908
Epoch 267
Validation binary_cross_entropy = 2.167374
Epoch 268
Validation binary_cross_entropy = 2.169666
Epoch 269
Loss = 8.0958e-05, PNorm = 93.5035, GNorm = 0.0025, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 2.172229
Epoch 270
Validation binary_cross_entropy = 2.174657
Epoch 271
Validation binary_cross_entropy = 2.176610
Epoch 272
Validation binary_cross_entropy = 2.178342
Epoch 273
Validation binary_cross_entropy = 2.180381
Epoch 274
Loss = 5.2564e-05, PNorm = 93.5062, GNorm = 0.0018, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 2.182171
Epoch 275
Validation binary_cross_entropy = 2.183702
Epoch 276
Validation binary_cross_entropy = 2.185084
Epoch 277
Validation binary_cross_entropy = 2.186867
Epoch 278
Validation binary_cross_entropy = 2.188945
Epoch 279
Loss = 5.6180e-05, PNorm = 93.5091, GNorm = 0.0017, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 2.190801
Epoch 280
Validation binary_cross_entropy = 2.192618
Epoch 281
Validation binary_cross_entropy = 2.194357
Epoch 282
Validation binary_cross_entropy = 2.195839
Epoch 283
Validation binary_cross_entropy = 2.197380
Epoch 284
Loss = 4.4876e-05, PNorm = 93.5117, GNorm = 0.0011, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 2.198753
Epoch 285
Validation binary_cross_entropy = 2.200130
Epoch 286
Validation binary_cross_entropy = 2.201533
Epoch 287
Validation binary_cross_entropy = 2.203264
Epoch 288
Validation binary_cross_entropy = 2.205611
Epoch 289
Loss = 4.4699e-05, PNorm = 93.5145, GNorm = 0.0011, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 2.207619
Epoch 290
Validation binary_cross_entropy = 2.209323
Epoch 291
Validation binary_cross_entropy = 2.210929
Epoch 292
Validation binary_cross_entropy = 2.212995
Epoch 293
Validation binary_cross_entropy = 2.215106
Epoch 294
Loss = 3.9296e-05, PNorm = 93.5175, GNorm = 0.0006, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 2.217097
Epoch 295
Validation binary_cross_entropy = 2.219007
Epoch 296
Validation binary_cross_entropy = 2.220758
Epoch 297
Validation binary_cross_entropy = 2.222400
Epoch 298
Validation binary_cross_entropy = 2.223739
Epoch 299
Loss = 7.2739e-05, PNorm = 93.5202, GNorm = 0.0017, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 2.225474
Model 0 best validation binary_cross_entropy = 0.256286 on epoch 46
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.183475
Ensemble test binary_cross_entropy = 0.183475
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1700, bias=False)
        (W_h): Linear(in_features=1700, out_features=1700, bias=False)
        (W_o): Linear(in_features=1833, out_features=1700, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=1700, out_features=1700, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=1700, out_features=1700, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.0, inplace=False)
    (7): Linear(in_features=1700, out_features=1, bias=True)
  )
)
Number of parameters = 12,042,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.560832
Epoch 1
Validation binary_cross_entropy = 0.694903
Epoch 2
Validation binary_cross_entropy = 0.669818
Epoch 3
Validation binary_cross_entropy = 0.754302
Epoch 4
Loss = 6.3105e-01, PNorm = 84.5257, GNorm = 0.3706, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.575267
Epoch 5
Validation binary_cross_entropy = 2.566610
Epoch 6
Validation binary_cross_entropy = 0.298012
Epoch 7
Validation binary_cross_entropy = 0.519717
Epoch 8
Validation binary_cross_entropy = 0.673536
Epoch 9
Loss = 6.7850e-01, PNorm = 84.8029, GNorm = 0.1678, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.678858
Epoch 10
Validation binary_cross_entropy = 0.613739
Epoch 11
Validation binary_cross_entropy = 0.476458
Epoch 12
Validation binary_cross_entropy = 0.515532
Epoch 13
Validation binary_cross_entropy = 0.603139
Epoch 14
Loss = 5.8671e-01, PNorm = 85.2617, GNorm = 2.2027, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.324119
Epoch 15
Validation binary_cross_entropy = 0.475087
Epoch 16
Validation binary_cross_entropy = 0.586473
Epoch 17
Validation binary_cross_entropy = 0.511555
Epoch 18
Validation binary_cross_entropy = 0.611862
Epoch 19
Loss = 5.0765e-01, PNorm = 85.6402, GNorm = 2.1772, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.481293
Epoch 20
Validation binary_cross_entropy = 0.397729
Epoch 21
Validation binary_cross_entropy = 0.415423
Epoch 22
Validation binary_cross_entropy = 0.395005
Epoch 23
Validation binary_cross_entropy = 0.399953
Epoch 24
Loss = 4.2784e-01, PNorm = 86.0255, GNorm = 2.1226, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.379135
Epoch 25
Validation binary_cross_entropy = 0.420081
Epoch 26
Validation binary_cross_entropy = 0.277915
Epoch 27
Validation binary_cross_entropy = 0.450868
Epoch 28
Validation binary_cross_entropy = 0.330253
Epoch 29
Loss = 3.4259e-01, PNorm = 86.3799, GNorm = 0.8357, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.525574
Epoch 30
Validation binary_cross_entropy = 0.432765
Epoch 31
Validation binary_cross_entropy = 0.391244
Epoch 32
Validation binary_cross_entropy = 0.330632
Epoch 33
Validation binary_cross_entropy = 0.682626
Epoch 34
Loss = 3.3922e-01, PNorm = 86.7364, GNorm = 0.7314, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.270219
Epoch 35
Validation binary_cross_entropy = 0.338165
Epoch 36
Validation binary_cross_entropy = 0.432925
Epoch 37
Validation binary_cross_entropy = 0.359807
Epoch 38
Validation binary_cross_entropy = 0.346145
Epoch 39
Loss = 2.5952e-01, PNorm = 87.1023, GNorm = 1.0517, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.387617
Epoch 40
Validation binary_cross_entropy = 0.467811
Epoch 41
Validation binary_cross_entropy = 0.411646
Epoch 42
Validation binary_cross_entropy = 0.387665
Epoch 43
Validation binary_cross_entropy = 0.502603
Epoch 44
Loss = 1.6942e-01, PNorm = 87.4735, GNorm = 1.0317, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.437300
Epoch 45
Validation binary_cross_entropy = 0.538483
Epoch 46
Validation binary_cross_entropy = 0.551173
Epoch 47
Validation binary_cross_entropy = 0.482227
Epoch 48
Validation binary_cross_entropy = 0.622473
Epoch 49
Loss = 1.9880e-01, PNorm = 87.7575, GNorm = 3.3488, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.694044
Epoch 50
Validation binary_cross_entropy = 0.463685
Epoch 51
Validation binary_cross_entropy = 0.436081
Epoch 52
Validation binary_cross_entropy = 0.627312
Epoch 53
Validation binary_cross_entropy = 0.451417
Epoch 54
Loss = 1.9540e-01, PNorm = 88.0726, GNorm = 1.7462, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.389622
Epoch 55
Validation binary_cross_entropy = 0.376584
Epoch 56
Validation binary_cross_entropy = 0.390614
Epoch 57
Validation binary_cross_entropy = 0.407401
Epoch 58
Validation binary_cross_entropy = 0.458904
Epoch 59
Loss = 7.9706e-02, PNorm = 88.4203, GNorm = 0.3364, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.536248
Epoch 60
Validation binary_cross_entropy = 0.595112
Epoch 61
Validation binary_cross_entropy = 0.671671
Epoch 62
Validation binary_cross_entropy = 0.683519
Epoch 63
Validation binary_cross_entropy = 0.713324
Epoch 64
Loss = 1.0830e-01, PNorm = 88.7244, GNorm = 0.7714, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.742330
Epoch 65
Validation binary_cross_entropy = 0.757675
Epoch 66
Validation binary_cross_entropy = 0.742956
Epoch 67
Validation binary_cross_entropy = 0.760356
Epoch 68
Validation binary_cross_entropy = 0.770781
Epoch 69
Loss = 1.1902e-01, PNorm = 88.9430, GNorm = 0.6985, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.793915
Epoch 70
Validation binary_cross_entropy = 0.778717
Epoch 71
Validation binary_cross_entropy = 0.746929
Epoch 72
Validation binary_cross_entropy = 0.741184
Epoch 73
Validation binary_cross_entropy = 0.732832
Epoch 74
Loss = 1.3682e-01, PNorm = 89.1180, GNorm = 2.4077, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.728453
Epoch 75
Validation binary_cross_entropy = 0.742688
Epoch 76
Validation binary_cross_entropy = 0.726584
Epoch 77
Validation binary_cross_entropy = 0.737461
Epoch 78
Validation binary_cross_entropy = 0.699970
Epoch 79
Loss = 8.8173e-02, PNorm = 89.2892, GNorm = 1.7565, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.732046
Epoch 80
Validation binary_cross_entropy = 0.731938
Epoch 81
Validation binary_cross_entropy = 0.728282
Epoch 82
Validation binary_cross_entropy = 0.812797
Epoch 83
Validation binary_cross_entropy = 0.854597
Epoch 84
Loss = 1.4318e-02, PNorm = 89.4900, GNorm = 0.2971, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.888634
Epoch 85
Validation binary_cross_entropy = 0.913929
Epoch 86
Validation binary_cross_entropy = 0.935238
Epoch 87
Validation binary_cross_entropy = 0.894353
Epoch 88
Validation binary_cross_entropy = 0.865520
Epoch 89
Loss = 6.7674e-02, PNorm = 89.6816, GNorm = 1.9653, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.944075
Epoch 90
Validation binary_cross_entropy = 0.881085
Epoch 91
Validation binary_cross_entropy = 0.859272
Epoch 92
Validation binary_cross_entropy = 0.873790
Epoch 93
Validation binary_cross_entropy = 0.809554
Epoch 94
Loss = 2.4409e-02, PNorm = 89.8341, GNorm = 1.0196, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.794691
Epoch 95
Validation binary_cross_entropy = 0.832569
Epoch 96
Validation binary_cross_entropy = 0.820819
Epoch 97
Validation binary_cross_entropy = 0.808339
Epoch 98
Validation binary_cross_entropy = 0.825781
Epoch 99
Loss = 2.1091e-02, PNorm = 89.9967, GNorm = 0.1436, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.923602
Epoch 100
Validation binary_cross_entropy = 0.982094
Epoch 101
Validation binary_cross_entropy = 0.988617
Epoch 102
Validation binary_cross_entropy = 1.062590
Epoch 103
Validation binary_cross_entropy = 1.090144
Epoch 104
Loss = 1.2733e-02, PNorm = 90.1495, GNorm = 0.2636, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 1.078577
Epoch 105
Validation binary_cross_entropy = 1.091920
Epoch 106
Validation binary_cross_entropy = 1.219555
Epoch 107
Validation binary_cross_entropy = 1.275887
Epoch 108
Validation binary_cross_entropy = 1.301337
Epoch 109
Loss = 7.5241e-03, PNorm = 90.2659, GNorm = 0.4905, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 1.354386
Epoch 110
Validation binary_cross_entropy = 1.331566
Epoch 111
Validation binary_cross_entropy = 1.475245
Epoch 112
Validation binary_cross_entropy = 1.118162
Epoch 113
Validation binary_cross_entropy = 0.999026
Epoch 114
Loss = 7.9588e-03, PNorm = 90.4336, GNorm = 0.1980, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.899728
Epoch 115
Validation binary_cross_entropy = 0.881032
Epoch 116
Validation binary_cross_entropy = 0.915449
Epoch 117
Validation binary_cross_entropy = 0.954374
Epoch 118
Validation binary_cross_entropy = 1.014006
Epoch 119
Loss = 2.3450e-02, PNorm = 90.6326, GNorm = 0.5100, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 1.045759
Epoch 120
Validation binary_cross_entropy = 1.057376
Epoch 121
Validation binary_cross_entropy = 1.100721
Epoch 122
Validation binary_cross_entropy = 1.122292
Epoch 123
Validation binary_cross_entropy = 1.128360
Epoch 124
Loss = 1.4347e-02, PNorm = 90.8093, GNorm = 0.0364, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 1.133718
Epoch 125
Validation binary_cross_entropy = 1.174809
Epoch 126
Validation binary_cross_entropy = 1.171027
Epoch 127
Validation binary_cross_entropy = 1.144100
Epoch 128
Validation binary_cross_entropy = 1.151885
Epoch 129
Loss = 2.5444e-03, PNorm = 90.9304, GNorm = 0.1871, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 1.147278
Epoch 130
Validation binary_cross_entropy = 1.140008
Epoch 131
Validation binary_cross_entropy = 1.076458
Epoch 132
Validation binary_cross_entropy = 1.044693
Epoch 133
Validation binary_cross_entropy = 1.009771
Epoch 134
Loss = 2.7707e-02, PNorm = 91.0200, GNorm = 0.3417, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 1.084013
Epoch 135
Validation binary_cross_entropy = 1.041418
Epoch 136
Validation binary_cross_entropy = 1.056561
Epoch 137
Validation binary_cross_entropy = 1.113085
Epoch 138
Validation binary_cross_entropy = 1.140358
Epoch 139
Loss = 4.7509e-02, PNorm = 91.1489, GNorm = 0.3424, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 1.140412
Epoch 140
Validation binary_cross_entropy = 1.246303
Epoch 141
Validation binary_cross_entropy = 1.274376
Epoch 142
Validation binary_cross_entropy = 1.314021
Epoch 143
Validation binary_cross_entropy = 1.367627
Epoch 144
Loss = 9.6727e-02, PNorm = 91.2846, GNorm = 2.9713, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.717604
Epoch 145
Validation binary_cross_entropy = 1.839527
Epoch 146
Validation binary_cross_entropy = 1.503794
Epoch 147
Validation binary_cross_entropy = 0.750268
Epoch 148
Validation binary_cross_entropy = 0.591067
Epoch 149
Loss = 1.4134e-01, PNorm = 91.5953, GNorm = 2.0221, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.576882
Epoch 150
Validation binary_cross_entropy = 0.523854
Epoch 151
Validation binary_cross_entropy = 0.499818
Epoch 152
Validation binary_cross_entropy = 0.509005
Epoch 153
Validation binary_cross_entropy = 0.506130
Epoch 154
Loss = 8.3541e-02, PNorm = 91.9867, GNorm = 0.9562, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 0.523006
Epoch 155
Validation binary_cross_entropy = 0.565639
Epoch 156
Validation binary_cross_entropy = 0.610926
Epoch 157
Validation binary_cross_entropy = 0.650830
Epoch 158
Validation binary_cross_entropy = 0.716090
Epoch 159
Loss = 5.4571e-02, PNorm = 92.2813, GNorm = 0.3668, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 0.760307
Epoch 160
Validation binary_cross_entropy = 0.797799
Epoch 161
Validation binary_cross_entropy = 0.834888
Epoch 162
Validation binary_cross_entropy = 0.866710
Epoch 163
Validation binary_cross_entropy = 0.875805
Epoch 164
Loss = 5.7778e-02, PNorm = 92.4820, GNorm = 0.6514, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 0.871116
Epoch 165
Validation binary_cross_entropy = 0.873630
Epoch 166
Validation binary_cross_entropy = 0.888535
Epoch 167
Validation binary_cross_entropy = 0.918421
Epoch 168
Validation binary_cross_entropy = 0.955581
Epoch 169
Loss = 1.8614e-02, PNorm = 92.6138, GNorm = 0.2890, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.000596
Epoch 170
Validation binary_cross_entropy = 1.027339
Epoch 171
Validation binary_cross_entropy = 1.040565
Epoch 172
Validation binary_cross_entropy = 1.035797
Epoch 173
Validation binary_cross_entropy = 1.054111
Epoch 174
Loss = 7.1082e-03, PNorm = 92.7303, GNorm = 0.1515, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.073010
Epoch 175
Validation binary_cross_entropy = 1.090338
Epoch 176
Validation binary_cross_entropy = 1.107352
Epoch 177
Validation binary_cross_entropy = 1.136946
Epoch 178
Validation binary_cross_entropy = 1.167153
Epoch 179
Loss = 1.2618e-02, PNorm = 92.8306, GNorm = 0.2163, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.125499
Epoch 180
Validation binary_cross_entropy = 1.107819
Epoch 181
Validation binary_cross_entropy = 1.123367
Epoch 182
Validation binary_cross_entropy = 1.182669
Epoch 183
Validation binary_cross_entropy = 1.239146
Epoch 184
Loss = 7.4584e-03, PNorm = 92.9287, GNorm = 0.1535, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.291190
Epoch 185
Validation binary_cross_entropy = 1.326991
Epoch 186
Validation binary_cross_entropy = 1.373737
Epoch 187
Validation binary_cross_entropy = 1.423408
Epoch 188
Validation binary_cross_entropy = 1.443749
Epoch 189
Loss = 2.8946e-03, PNorm = 93.0144, GNorm = 0.1105, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.451060
Epoch 190
Validation binary_cross_entropy = 1.456869
Epoch 191
Validation binary_cross_entropy = 1.497372
Epoch 192
Validation binary_cross_entropy = 1.533778
Epoch 193
Validation binary_cross_entropy = 1.568534
Epoch 194
Loss = 1.1759e-03, PNorm = 93.0768, GNorm = 0.0419, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.604414
Epoch 195
Validation binary_cross_entropy = 1.640734
Epoch 196
Validation binary_cross_entropy = 1.677064
Epoch 197
Validation binary_cross_entropy = 1.720984
Epoch 198
Validation binary_cross_entropy = 1.752606
Epoch 199
Loss = 5.6677e-04, PNorm = 93.1213, GNorm = 0.0254, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.776239
Epoch 200
Validation binary_cross_entropy = 1.823926
Epoch 201
Validation binary_cross_entropy = 1.756285
Epoch 202
Validation binary_cross_entropy = 1.640634
Epoch 203
Validation binary_cross_entropy = 1.584076
Epoch 204
Loss = 6.6290e-03, PNorm = 93.1459, GNorm = 0.8511, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.547074
Epoch 205
Validation binary_cross_entropy = 1.531648
Epoch 206
Validation binary_cross_entropy = 1.527381
Epoch 207
Validation binary_cross_entropy = 1.527553
Epoch 208
Validation binary_cross_entropy = 1.536328
Epoch 209
Loss = 3.8588e-03, PNorm = 93.1950, GNorm = 0.2133, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.546793
Epoch 210
Validation binary_cross_entropy = 1.551665
Epoch 211
Validation binary_cross_entropy = 1.578201
Epoch 212
Validation binary_cross_entropy = 1.629047
Epoch 213
Validation binary_cross_entropy = 1.696205
Epoch 214
Loss = 1.9355e-02, PNorm = 93.2398, GNorm = 1.6110, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.683321
Epoch 215
Validation binary_cross_entropy = 1.614049
Epoch 216
Validation binary_cross_entropy = 1.582275
Epoch 217
Validation binary_cross_entropy = 1.575207
Epoch 218
Validation binary_cross_entropy = 1.559215
Epoch 219
Loss = 2.3827e-02, PNorm = 93.2829, GNorm = 0.5271, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.534089
Epoch 220
Validation binary_cross_entropy = 1.585377
Epoch 221
Validation binary_cross_entropy = 1.612074
Epoch 222
Validation binary_cross_entropy = 1.493896
Epoch 223
Validation binary_cross_entropy = 1.467297
Epoch 224
Loss = 1.4732e-02, PNorm = 93.3607, GNorm = 1.1451, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.495684
Epoch 225
Validation binary_cross_entropy = 1.552134
Epoch 226
Validation binary_cross_entropy = 1.589478
Epoch 227
Validation binary_cross_entropy = 1.607531
Epoch 228
Validation binary_cross_entropy = 1.623379
Epoch 229
Loss = 3.3212e-03, PNorm = 93.4645, GNorm = 0.1570, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.634650
Epoch 230
Validation binary_cross_entropy = 1.620054
Epoch 231
Validation binary_cross_entropy = 1.614908
Epoch 232
Validation binary_cross_entropy = 1.652162
Epoch 233
Validation binary_cross_entropy = 1.675490
Epoch 234
Loss = 1.3449e-02, PNorm = 93.5575, GNorm = 0.9663, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.671646
Epoch 235
Validation binary_cross_entropy = 1.637599
Epoch 236
Validation binary_cross_entropy = 1.614077
Epoch 237
Validation binary_cross_entropy = 1.605569
Epoch 238
Validation binary_cross_entropy = 1.612048
Epoch 239
Loss = 4.0960e-02, PNorm = 93.6181, GNorm = 1.5858, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.810583
Epoch 240
Validation binary_cross_entropy = 1.813062
Epoch 241
Validation binary_cross_entropy = 1.717170
Epoch 242
Validation binary_cross_entropy = 1.652264
Epoch 243
Validation binary_cross_entropy = 1.605482
Epoch 244
Loss = 4.5280e-02, PNorm = 93.6607, GNorm = 0.1013, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.578552
Epoch 245
Validation binary_cross_entropy = 1.563341
Epoch 246
Validation binary_cross_entropy = 1.542490
Epoch 247
Validation binary_cross_entropy = 1.537393
Epoch 248
Validation binary_cross_entropy = 1.528913
Epoch 249
Loss = 2.4253e-03, PNorm = 93.7196, GNorm = 0.0718, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.502880
Epoch 250
Validation binary_cross_entropy = 1.457996
Epoch 251
Validation binary_cross_entropy = 1.409945
Epoch 252
Validation binary_cross_entropy = 1.382222
Epoch 253
Validation binary_cross_entropy = 1.455647
Epoch 254
Loss = 1.1131e-03, PNorm = 93.7698, GNorm = 0.0477, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.517734
Epoch 255
Validation binary_cross_entropy = 1.566462
Epoch 256
Validation binary_cross_entropy = 1.601337
Epoch 257
Validation binary_cross_entropy = 1.624516
Epoch 258
Validation binary_cross_entropy = 1.641189
Epoch 259
Loss = 5.3694e-04, PNorm = 93.8041, GNorm = 0.0206, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.657722
Epoch 260
Validation binary_cross_entropy = 1.672874
Epoch 261
Validation binary_cross_entropy = 1.697002
Epoch 262
Validation binary_cross_entropy = 1.721634
Epoch 263
Validation binary_cross_entropy = 1.745897
Epoch 264
Loss = 1.2268e-03, PNorm = 93.8322, GNorm = 0.0194, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.769658
Epoch 265
Validation binary_cross_entropy = 1.785762
Epoch 266
Validation binary_cross_entropy = 1.791860
Epoch 267
Validation binary_cross_entropy = 1.839580
Epoch 268
Validation binary_cross_entropy = 1.852219
Epoch 269
Loss = 5.6368e-03, PNorm = 93.8583, GNorm = 0.3978, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.822581
Epoch 270
Validation binary_cross_entropy = 1.782054
Epoch 271
Validation binary_cross_entropy = 1.745113
Epoch 272
Validation binary_cross_entropy = 1.715656
Epoch 273
Validation binary_cross_entropy = 1.695324
Epoch 274
Loss = 1.8540e-03, PNorm = 93.8871, GNorm = 0.2933, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.690649
Epoch 275
Validation binary_cross_entropy = 1.750031
Epoch 276
Validation binary_cross_entropy = 1.819296
Epoch 277
Validation binary_cross_entropy = 1.790206
Epoch 278
Validation binary_cross_entropy = 1.776207
Epoch 279
Loss = 4.4239e-04, PNorm = 93.9232, GNorm = 0.0149, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.774577
Epoch 280
Validation binary_cross_entropy = 1.779083
Epoch 281
Validation binary_cross_entropy = 1.783609
Epoch 282
Validation binary_cross_entropy = 1.787732
Epoch 283
Validation binary_cross_entropy = 1.800780
Epoch 284
Loss = 1.1728e-04, PNorm = 93.9435, GNorm = 0.0024, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.821337
Epoch 285
Validation binary_cross_entropy = 1.838192
Epoch 286
Validation binary_cross_entropy = 1.851759
Epoch 287
Validation binary_cross_entropy = 1.876196
Epoch 288
Validation binary_cross_entropy = 1.901275
Epoch 289
Loss = 3.3764e-03, PNorm = 93.9641, GNorm = 0.1262, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.897391
Epoch 290
Validation binary_cross_entropy = 1.889233
Epoch 291
Validation binary_cross_entropy = 1.884823
Epoch 292
Validation binary_cross_entropy = 1.881856
Epoch 293
Validation binary_cross_entropy = 1.878445
Epoch 294
Loss = 6.5450e-04, PNorm = 93.9828, GNorm = 0.0444, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.875245
Epoch 295
Validation binary_cross_entropy = 1.906317
Epoch 296
Validation binary_cross_entropy = 1.961534
Epoch 297
Validation binary_cross_entropy = 2.001298
Epoch 298
Validation binary_cross_entropy = 2.012490
Epoch 299
Loss = 1.2521e-03, PNorm = 94.0016, GNorm = 0.1189, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 2.001818
Model 0 best validation binary_cross_entropy = 0.270219 on epoch 34
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.171303
Ensemble test binary_cross_entropy = 0.171303
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1700, bias=False)
        (W_h): Linear(in_features=1700, out_features=1700, bias=False)
        (W_o): Linear(in_features=1833, out_features=1700, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=1700, out_features=1700, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=1700, out_features=1700, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.0, inplace=False)
    (7): Linear(in_features=1700, out_features=1, bias=True)
  )
)
Number of parameters = 12,042,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.567432
Epoch 1
Validation binary_cross_entropy = 0.695972
Epoch 2
Validation binary_cross_entropy = 0.635688
Epoch 3
Validation binary_cross_entropy = 0.757854
Epoch 4
Loss = 6.8667e-01, PNorm = 84.5312, GNorm = 2.2644, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 1.312171
Epoch 5
Validation binary_cross_entropy = 0.384556
Epoch 6
Validation binary_cross_entropy = 0.657260
Epoch 7
Validation binary_cross_entropy = 0.682134
Epoch 8
Validation binary_cross_entropy = 0.571622
Epoch 9
Loss = 5.9498e-01, PNorm = 84.8160, GNorm = 0.9477, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.363365
Epoch 10
Validation binary_cross_entropy = 0.566232
Epoch 11
Validation binary_cross_entropy = 0.398818
Epoch 12
Validation binary_cross_entropy = 0.534641
Epoch 13
Validation binary_cross_entropy = 0.304075
Epoch 14
Loss = 4.9081e-01, PNorm = 85.2062, GNorm = 0.4926, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.743125
Epoch 15
Validation binary_cross_entropy = 0.269347
Epoch 16
Validation binary_cross_entropy = 0.602684
Epoch 17
Validation binary_cross_entropy = 0.461972
Epoch 18
Validation binary_cross_entropy = 0.378207
Epoch 19
Loss = 4.0357e-01, PNorm = 85.5896, GNorm = 0.4845, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.596418
Epoch 20
Validation binary_cross_entropy = 0.306571
Epoch 21
Validation binary_cross_entropy = 0.379519
Epoch 22
Validation binary_cross_entropy = 0.343191
Epoch 23
Validation binary_cross_entropy = 0.353528
Epoch 24
Loss = 3.6875e-01, PNorm = 85.9690, GNorm = 1.0233, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.442113
Epoch 25
Validation binary_cross_entropy = 0.262964
Epoch 26
Validation binary_cross_entropy = 0.618822
Epoch 27
Validation binary_cross_entropy = 0.277954
Epoch 28
Validation binary_cross_entropy = 0.370643
Epoch 29
Loss = 4.0644e-01, PNorm = 86.3261, GNorm = 0.4373, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.318332
Epoch 30
Validation binary_cross_entropy = 0.424057
Epoch 31
Validation binary_cross_entropy = 0.351242
Epoch 32
Validation binary_cross_entropy = 0.372090
Epoch 33
Validation binary_cross_entropy = 0.399443
Epoch 34
Loss = 2.6204e-01, PNorm = 86.7423, GNorm = 1.0478, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.376289
Epoch 35
Validation binary_cross_entropy = 0.689650
Epoch 36
Validation binary_cross_entropy = 0.357751
Epoch 37
Validation binary_cross_entropy = 0.500015
Epoch 38
Validation binary_cross_entropy = 0.337785
Epoch 39
Loss = 2.7578e-01, PNorm = 87.0985, GNorm = 0.4477, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.269907
Epoch 40
Validation binary_cross_entropy = 0.347114
Epoch 41
Validation binary_cross_entropy = 0.444086
Epoch 42
Validation binary_cross_entropy = 0.387304
Epoch 43
Validation binary_cross_entropy = 0.341428
Epoch 44
Loss = 3.0491e-01, PNorm = 87.4989, GNorm = 1.0398, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.465934
Epoch 45
Validation binary_cross_entropy = 0.422583
Epoch 46
Validation binary_cross_entropy = 0.408923
Epoch 47
Validation binary_cross_entropy = 0.504152
Epoch 48
Validation binary_cross_entropy = 0.545688
Epoch 49
Loss = 1.8507e-01, PNorm = 87.8800, GNorm = 2.9958, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.550027
Epoch 50
Validation binary_cross_entropy = 0.621489
Epoch 51
Validation binary_cross_entropy = 0.621876
Epoch 52
Validation binary_cross_entropy = 0.586547
Epoch 53
Validation binary_cross_entropy = 0.578619
Epoch 54
Loss = 1.0358e-01, PNorm = 88.2168, GNorm = 0.5645, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.585059
Epoch 55
Validation binary_cross_entropy = 0.582201
Epoch 56
Validation binary_cross_entropy = 0.569835
Epoch 57
Validation binary_cross_entropy = 0.595104
Epoch 58
Validation binary_cross_entropy = 0.608937
Epoch 59
Loss = 1.2278e-01, PNorm = 88.5014, GNorm = 0.1176, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.585532
Epoch 60
Validation binary_cross_entropy = 0.586997
Epoch 61
Validation binary_cross_entropy = 0.581386
Epoch 62
Validation binary_cross_entropy = 0.582348
Epoch 63
Validation binary_cross_entropy = 0.621513
Epoch 64
Loss = 9.0646e-02, PNorm = 88.7780, GNorm = 0.4466, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.645169
Epoch 65
Validation binary_cross_entropy = 0.708371
Epoch 66
Validation binary_cross_entropy = 0.724390
Epoch 67
Validation binary_cross_entropy = 0.725675
Epoch 68
Validation binary_cross_entropy = 0.857885
Epoch 69
Loss = 2.9775e-01, PNorm = 89.0522, GNorm = 2.7371, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.716152
Epoch 70
Validation binary_cross_entropy = 0.712604
Epoch 71
Validation binary_cross_entropy = 0.586096
Epoch 72
Validation binary_cross_entropy = 0.813555
Epoch 73
Validation binary_cross_entropy = 0.567793
Epoch 74
Loss = 1.7942e-01, PNorm = 89.3801, GNorm = 0.7316, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.598527
Epoch 75
Validation binary_cross_entropy = 0.459746
Epoch 76
Validation binary_cross_entropy = 0.405463
Epoch 77
Validation binary_cross_entropy = 0.427533
Epoch 78
Validation binary_cross_entropy = 0.459092
Epoch 79
Loss = 1.7550e-01, PNorm = 89.7407, GNorm = 0.4174, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.501495
Epoch 80
Validation binary_cross_entropy = 0.554036
Epoch 81
Validation binary_cross_entropy = 0.601720
Epoch 82
Validation binary_cross_entropy = 0.682531
Epoch 83
Validation binary_cross_entropy = 0.752718
Epoch 84
Loss = 9.4088e-02, PNorm = 90.0563, GNorm = 1.1722, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.818148
Epoch 85
Validation binary_cross_entropy = 0.872679
Epoch 86
Validation binary_cross_entropy = 0.905143
Epoch 87
Validation binary_cross_entropy = 0.961488
Epoch 88
Validation binary_cross_entropy = 1.011916
Epoch 89
Loss = 8.1012e-02, PNorm = 90.2792, GNorm = 1.0718, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 1.011750
Epoch 90
Validation binary_cross_entropy = 1.066839
Epoch 91
Validation binary_cross_entropy = 1.036981
Epoch 92
Validation binary_cross_entropy = 1.019773
Epoch 93
Validation binary_cross_entropy = 1.021209
Epoch 94
Loss = 5.0030e-02, PNorm = 90.4178, GNorm = 0.4089, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.022741
Epoch 95
Validation binary_cross_entropy = 1.107491
Epoch 96
Validation binary_cross_entropy = 0.978540
Epoch 97
Validation binary_cross_entropy = 0.920952
Epoch 98
Validation binary_cross_entropy = 0.902836
Epoch 99
Loss = 2.4555e-02, PNorm = 90.5925, GNorm = 0.4314, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.924746
Epoch 100
Validation binary_cross_entropy = 0.947401
Epoch 101
Validation binary_cross_entropy = 1.001772
Epoch 102
Validation binary_cross_entropy = 1.092228
Epoch 103
Validation binary_cross_entropy = 1.090342
Epoch 104
Loss = 7.1182e-02, PNorm = 90.7487, GNorm = 0.7941, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 1.055726
Epoch 105
Validation binary_cross_entropy = 1.039934
Epoch 106
Validation binary_cross_entropy = 1.012589
Epoch 107
Validation binary_cross_entropy = 0.988576
Epoch 108
Validation binary_cross_entropy = 0.978472
Epoch 109
Loss = 1.5965e-02, PNorm = 90.8786, GNorm = 0.4207, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.984330
Epoch 110
Validation binary_cross_entropy = 0.985511
Epoch 111
Validation binary_cross_entropy = 1.012278
Epoch 112
Validation binary_cross_entropy = 1.088469
Epoch 113
Validation binary_cross_entropy = 1.080793
Epoch 114
Loss = 3.8067e-03, PNorm = 90.9881, GNorm = 0.1306, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 1.082710
Epoch 115
Validation binary_cross_entropy = 1.059148
Epoch 116
Validation binary_cross_entropy = 1.161306
Epoch 117
Validation binary_cross_entropy = 1.094007
Epoch 118
Validation binary_cross_entropy = 1.097293
Epoch 119
Loss = 2.9032e-03, PNorm = 91.1427, GNorm = 0.2261, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 1.084147
Epoch 120
Validation binary_cross_entropy = 1.101686
Epoch 121
Validation binary_cross_entropy = 1.147583
Epoch 122
Validation binary_cross_entropy = 1.173774
Epoch 123
Validation binary_cross_entropy = 1.133358
Epoch 124
Loss = 1.8823e-02, PNorm = 91.3153, GNorm = 0.0222, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 1.091067
Epoch 125
Validation binary_cross_entropy = 1.042370
Epoch 126
Validation binary_cross_entropy = 1.042399
Epoch 127
Validation binary_cross_entropy = 0.965860
Epoch 128
Validation binary_cross_entropy = 0.969648
Epoch 129
Loss = 5.5165e-02, PNorm = 91.4910, GNorm = 0.0225, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 1.019969
Epoch 130
Validation binary_cross_entropy = 0.861955
Epoch 131
Validation binary_cross_entropy = 0.859646
Epoch 132
Validation binary_cross_entropy = 0.874896
Epoch 133
Validation binary_cross_entropy = 0.902362
Epoch 134
Loss = 1.1735e-02, PNorm = 91.6850, GNorm = 0.0486, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.999554
Epoch 135
Validation binary_cross_entropy = 0.914644
Epoch 136
Validation binary_cross_entropy = 1.033466
Epoch 137
Validation binary_cross_entropy = 1.041211
Epoch 138
Validation binary_cross_entropy = 1.059587
Epoch 139
Loss = 2.8765e-02, PNorm = 91.8973, GNorm = 0.4968, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 1.105468
Epoch 140
Validation binary_cross_entropy = 1.196273
Epoch 141
Validation binary_cross_entropy = 1.224086
Epoch 142
Validation binary_cross_entropy = 1.224893
Epoch 143
Validation binary_cross_entropy = 1.243508
Epoch 144
Loss = 7.3501e-02, PNorm = 92.0488, GNorm = 0.1161, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.310985
Epoch 145
Validation binary_cross_entropy = 1.226722
Epoch 146
Validation binary_cross_entropy = 1.074969
Epoch 147
Validation binary_cross_entropy = 1.029782
Epoch 148
Validation binary_cross_entropy = 1.026041
Epoch 149
Loss = 2.6221e-02, PNorm = 92.1425, GNorm = 0.0835, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.022010
Epoch 150
Validation binary_cross_entropy = 1.092397
Epoch 151
Validation binary_cross_entropy = 1.149202
Epoch 152
Validation binary_cross_entropy = 1.162870
Epoch 153
Validation binary_cross_entropy = 1.180974
Epoch 154
Loss = 2.0268e-03, PNorm = 92.2299, GNorm = 0.0222, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 1.207960
Epoch 155
Validation binary_cross_entropy = 1.238398
Epoch 156
Validation binary_cross_entropy = 1.273997
Epoch 157
Validation binary_cross_entropy = 1.308775
Epoch 158
Validation binary_cross_entropy = 1.366431
Epoch 159
Loss = 3.1871e-02, PNorm = 92.2954, GNorm = 0.8976, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.447163
Epoch 160
Validation binary_cross_entropy = 1.428385
Epoch 161
Validation binary_cross_entropy = 1.385174
Epoch 162
Validation binary_cross_entropy = 1.363666
Epoch 163
Validation binary_cross_entropy = 1.368461
Epoch 164
Loss = 2.9178e-03, PNorm = 92.3370, GNorm = 0.2949, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.374609
Epoch 165
Validation binary_cross_entropy = 1.385740
Epoch 166
Validation binary_cross_entropy = 1.414659
Epoch 167
Validation binary_cross_entropy = 1.434172
Epoch 168
Validation binary_cross_entropy = 1.423437
Epoch 169
Loss = 1.8427e-03, PNorm = 92.3782, GNorm = 0.0820, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.412805
Epoch 170
Validation binary_cross_entropy = 1.414785
Epoch 171
Validation binary_cross_entropy = 1.423231
Epoch 172
Validation binary_cross_entropy = 1.435204
Epoch 173
Validation binary_cross_entropy = 1.449882
Epoch 174
Loss = 9.0267e-04, PNorm = 92.4127, GNorm = 0.0183, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.481613
Epoch 175
Validation binary_cross_entropy = 1.523351
Epoch 176
Validation binary_cross_entropy = 1.549536
Epoch 177
Validation binary_cross_entropy = 1.555526
Epoch 178
Validation binary_cross_entropy = 1.588507
Epoch 179
Loss = 2.8926e-03, PNorm = 92.4442, GNorm = 0.1436, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.646314
Epoch 180
Validation binary_cross_entropy = 1.653182
Epoch 181
Validation binary_cross_entropy = 1.636065
Epoch 182
Validation binary_cross_entropy = 1.609991
Epoch 183
Validation binary_cross_entropy = 1.589098
Epoch 184
Loss = 3.5198e-04, PNorm = 92.4765, GNorm = 0.0149, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.580783
Epoch 185
Validation binary_cross_entropy = 1.579985
Epoch 186
Validation binary_cross_entropy = 1.584142
Epoch 187
Validation binary_cross_entropy = 1.637549
Epoch 188
Validation binary_cross_entropy = 1.708113
Epoch 189
Loss = 3.2150e-04, PNorm = 92.5129, GNorm = 0.0066, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.774869
Epoch 190
Validation binary_cross_entropy = 1.825090
Epoch 191
Validation binary_cross_entropy = 1.836020
Epoch 192
Validation binary_cross_entropy = 1.828117
Epoch 193
Validation binary_cross_entropy = 1.817816
Epoch 194
Loss = 1.4031e-03, PNorm = 92.5402, GNorm = 0.1031, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.813162
Epoch 195
Validation binary_cross_entropy = 1.835160
Epoch 196
Validation binary_cross_entropy = 1.884076
Epoch 197
Validation binary_cross_entropy = 1.847053
Epoch 198
Validation binary_cross_entropy = 1.758839
Epoch 199
Loss = 2.1976e-04, PNorm = 92.5777, GNorm = 0.0033, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.703956
Epoch 200
Validation binary_cross_entropy = 1.700668
Epoch 201
Validation binary_cross_entropy = 1.738215
Epoch 202
Validation binary_cross_entropy = 1.756182
Epoch 203
Validation binary_cross_entropy = 1.870402
Epoch 204
Loss = 1.1760e-01, PNorm = 92.6123, GNorm = 4.0953, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.885072
Epoch 205
Validation binary_cross_entropy = 1.664731
Epoch 206
Validation binary_cross_entropy = 1.536934
Epoch 207
Validation binary_cross_entropy = 1.510853
Epoch 208
Validation binary_cross_entropy = 1.485807
Epoch 209
Loss = 7.8033e-03, PNorm = 92.6582, GNorm = 0.0474, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.562400
Epoch 210
Validation binary_cross_entropy = 1.576503
Epoch 211
Validation binary_cross_entropy = 1.537877
Epoch 212
Validation binary_cross_entropy = 1.494537
Epoch 213
Validation binary_cross_entropy = 1.448527
Epoch 214
Loss = 3.2201e-02, PNorm = 92.7204, GNorm = 0.8972, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.424180
Epoch 215
Validation binary_cross_entropy = 1.428037
Epoch 216
Validation binary_cross_entropy = 1.498071
Epoch 217
Validation binary_cross_entropy = 1.412021
Epoch 218
Validation binary_cross_entropy = 1.365432
Epoch 219
Loss = 8.2848e-04, PNorm = 92.7854, GNorm = 0.1237, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.395180
Epoch 220
Validation binary_cross_entropy = 1.424717
Epoch 221
Validation binary_cross_entropy = 1.660275
Epoch 222
Validation binary_cross_entropy = 1.413408
Epoch 223
Validation binary_cross_entropy = 1.313366
Epoch 224
Loss = 7.5474e-02, PNorm = 92.9229, GNorm = 1.1196, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.237605
Epoch 225
Validation binary_cross_entropy = 1.149733
Epoch 226
Validation binary_cross_entropy = 1.133999
Epoch 227
Validation binary_cross_entropy = 1.171082
Epoch 228
Validation binary_cross_entropy = 1.012176
Epoch 229
Loss = 6.2943e-03, PNorm = 93.0309, GNorm = 0.1453, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 0.974962
Epoch 230
Validation binary_cross_entropy = 1.022960
Epoch 231
Validation binary_cross_entropy = 1.045567
Epoch 232
Validation binary_cross_entropy = 1.082055
Epoch 233
Validation binary_cross_entropy = 1.143316
Epoch 234
Loss = 1.3831e-02, PNorm = 93.1741, GNorm = 0.1401, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.216340
Epoch 235
Validation binary_cross_entropy = 1.267313
Epoch 236
Validation binary_cross_entropy = 1.306036
Epoch 237
Validation binary_cross_entropy = 1.388079
Epoch 238
Validation binary_cross_entropy = 1.484680
Epoch 239
Loss = 1.3088e-02, PNorm = 93.2929, GNorm = 1.1966, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.521994
Epoch 240
Validation binary_cross_entropy = 1.517044
Epoch 241
Validation binary_cross_entropy = 1.510192
Epoch 242
Validation binary_cross_entropy = 1.506616
Epoch 243
Validation binary_cross_entropy = 1.509042
Epoch 244
Loss = 1.5518e-03, PNorm = 93.3703, GNorm = 0.0127, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.521287
Epoch 245
Validation binary_cross_entropy = 1.536765
Epoch 246
Validation binary_cross_entropy = 1.556359
Epoch 247
Validation binary_cross_entropy = 1.586638
Epoch 248
Validation binary_cross_entropy = 1.614040
Epoch 249
Loss = 1.3208e-03, PNorm = 93.4187, GNorm = 0.0893, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.637423
Epoch 250
Validation binary_cross_entropy = 1.657610
Epoch 251
Validation binary_cross_entropy = 1.677517
Epoch 252
Validation binary_cross_entropy = 1.697488
Epoch 253
Validation binary_cross_entropy = 1.715680
Epoch 254
Loss = 4.4104e-04, PNorm = 93.4483, GNorm = 0.0167, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.731328
Epoch 255
Validation binary_cross_entropy = 1.743909
Epoch 256
Validation binary_cross_entropy = 1.753503
Epoch 257
Validation binary_cross_entropy = 1.760764
Epoch 258
Validation binary_cross_entropy = 1.766403
Epoch 259
Loss = 3.0716e-04, PNorm = 93.4649, GNorm = 0.0085, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.771040
Epoch 260
Validation binary_cross_entropy = 1.774867
Epoch 261
Validation binary_cross_entropy = 1.785520
Epoch 262
Validation binary_cross_entropy = 1.795007
Epoch 263
Validation binary_cross_entropy = 1.802984
Epoch 264
Loss = 2.5175e-04, PNorm = 93.4780, GNorm = 0.0082, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.809347
Epoch 265
Validation binary_cross_entropy = 1.814492
Epoch 266
Validation binary_cross_entropy = 1.818675
Epoch 267
Validation binary_cross_entropy = 1.822000
Epoch 268
Validation binary_cross_entropy = 1.824569
Epoch 269
Loss = 2.6924e-04, PNorm = 93.4874, GNorm = 0.0083, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.827221
Epoch 270
Validation binary_cross_entropy = 1.829745
Epoch 271
Validation binary_cross_entropy = 1.831956
Epoch 272
Validation binary_cross_entropy = 1.834170
Epoch 273
Validation binary_cross_entropy = 1.836475
Epoch 274
Loss = 1.1959e-04, PNorm = 93.4950, GNorm = 0.0022, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.838497
Epoch 275
Validation binary_cross_entropy = 1.840342
Epoch 276
Validation binary_cross_entropy = 1.842107
Epoch 277
Validation binary_cross_entropy = 1.844432
Epoch 278
Validation binary_cross_entropy = 1.847761
Epoch 279
Loss = 3.0836e-04, PNorm = 93.5023, GNorm = 0.0181, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.851349
Epoch 280
Validation binary_cross_entropy = 1.855233
Epoch 281
Validation binary_cross_entropy = 1.859901
Epoch 282
Validation binary_cross_entropy = 1.866238
Epoch 283
Validation binary_cross_entropy = 1.872038
Epoch 284
Loss = 1.8064e-04, PNorm = 93.5119, GNorm = 0.0041, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.877353
Epoch 285
Validation binary_cross_entropy = 1.882069
Epoch 286
Validation binary_cross_entropy = 1.885895
Epoch 287
Validation binary_cross_entropy = 1.889026
Epoch 288
Validation binary_cross_entropy = 1.891632
Epoch 289
Loss = 1.2621e-04, PNorm = 93.5195, GNorm = 0.0030, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.893780
Epoch 290
Validation binary_cross_entropy = 1.895318
Epoch 291
Validation binary_cross_entropy = 1.896929
Epoch 292
Validation binary_cross_entropy = 1.898173
Epoch 293
Validation binary_cross_entropy = 1.899615
Epoch 294
Loss = 1.4827e-04, PNorm = 93.5257, GNorm = 0.0074, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.901726
Epoch 295
Validation binary_cross_entropy = 1.904555
Epoch 296
Validation binary_cross_entropy = 1.906792
Epoch 297
Validation binary_cross_entropy = 1.909073
Epoch 298
Validation binary_cross_entropy = 1.911535
Epoch 299
Loss = 1.4472e-04, PNorm = 93.5316, GNorm = 0.0067, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.914727
Model 0 best validation binary_cross_entropy = 0.262964 on epoch 25
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.177341
Ensemble test binary_cross_entropy = 0.177341
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1700, bias=False)
        (W_h): Linear(in_features=1700, out_features=1700, bias=False)
        (W_o): Linear(in_features=1833, out_features=1700, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=1700, out_features=1700, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=1700, out_features=1700, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.0, inplace=False)
    (7): Linear(in_features=1700, out_features=1, bias=True)
  )
)
Number of parameters = 12,042,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.547039
Epoch 1
Validation binary_cross_entropy = 0.742123
Epoch 2
Validation binary_cross_entropy = 0.628518
Epoch 3
Validation binary_cross_entropy = 0.679886
Epoch 4
Loss = 7.2681e-01, PNorm = 84.5208, GNorm = 3.0588, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 1.097584
Epoch 5
Validation binary_cross_entropy = 0.496712
Epoch 6
Validation binary_cross_entropy = 0.640334
Epoch 7
Validation binary_cross_entropy = 0.705554
Epoch 8
Validation binary_cross_entropy = 0.525184
Epoch 9
Loss = 7.3679e-01, PNorm = 84.8046, GNorm = 3.9719, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.931307
Epoch 10
Validation binary_cross_entropy = 0.314598
Epoch 11
Validation binary_cross_entropy = 0.951386
Epoch 12
Validation binary_cross_entropy = 0.758495
Epoch 13
Validation binary_cross_entropy = 0.552217
Epoch 14
Loss = 6.2329e-01, PNorm = 85.1553, GNorm = 0.3299, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.467090
Epoch 15
Validation binary_cross_entropy = 0.487010
Epoch 16
Validation binary_cross_entropy = 0.584406
Epoch 17
Validation binary_cross_entropy = 0.344222
Epoch 18
Validation binary_cross_entropy = 0.626965
Epoch 19
Loss = 4.2967e-01, PNorm = 85.5790, GNorm = 0.6701, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.384984
Epoch 20
Validation binary_cross_entropy = 0.428554
Epoch 21
Validation binary_cross_entropy = 0.325711
Epoch 22
Validation binary_cross_entropy = 0.565427
Epoch 23
Validation binary_cross_entropy = 0.299102
Epoch 24
Loss = 4.0956e-01, PNorm = 85.9857, GNorm = 2.0704, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.479125
Epoch 25
Validation binary_cross_entropy = 0.583535
Epoch 26
Validation binary_cross_entropy = 0.278459
Epoch 27
Validation binary_cross_entropy = 0.365851
Epoch 28
Validation binary_cross_entropy = 0.624582
Epoch 29
Loss = 4.8829e-01, PNorm = 86.3623, GNorm = 0.8282, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.403573
Epoch 30
Validation binary_cross_entropy = 0.275943
Epoch 31
Validation binary_cross_entropy = 0.351566
Epoch 32
Validation binary_cross_entropy = 0.365045
Epoch 33
Validation binary_cross_entropy = 0.308184
Epoch 34
Loss = 2.5126e-01, PNorm = 86.7748, GNorm = 0.3989, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.441152
Epoch 35
Validation binary_cross_entropy = 0.339952
Epoch 36
Validation binary_cross_entropy = 0.606543
Epoch 37
Validation binary_cross_entropy = 0.396508
Epoch 38
Validation binary_cross_entropy = 0.396379
Epoch 39
Loss = 2.3233e-01, PNorm = 87.2035, GNorm = 1.5440, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.479780
Epoch 40
Validation binary_cross_entropy = 0.524369
Epoch 41
Validation binary_cross_entropy = 0.532840
Epoch 42
Validation binary_cross_entropy = 0.418736
Epoch 43
Validation binary_cross_entropy = 0.926759
Epoch 44
Loss = 5.4278e-01, PNorm = 87.5816, GNorm = 2.9319, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.681519
Epoch 45
Validation binary_cross_entropy = 0.310439
Epoch 46
Validation binary_cross_entropy = 0.297836
Epoch 47
Validation binary_cross_entropy = 0.412199
Epoch 48
Validation binary_cross_entropy = 0.502421
Epoch 49
Loss = 3.2080e-01, PNorm = 88.0401, GNorm = 0.9547, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.415680
Epoch 50
Validation binary_cross_entropy = 0.373191
Epoch 51
Validation binary_cross_entropy = 0.377712
Epoch 52
Validation binary_cross_entropy = 0.434025
Epoch 53
Validation binary_cross_entropy = 0.442508
Epoch 54
Loss = 1.6458e-01, PNorm = 88.5004, GNorm = 0.6724, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.428627
Epoch 55
Validation binary_cross_entropy = 0.486882
Epoch 56
Validation binary_cross_entropy = 0.570128
Epoch 57
Validation binary_cross_entropy = 0.628719
Epoch 58
Validation binary_cross_entropy = 0.695134
Epoch 59
Loss = 1.1462e-01, PNorm = 88.9213, GNorm = 1.2457, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.691363
Epoch 60
Validation binary_cross_entropy = 0.762700
Epoch 61
Validation binary_cross_entropy = 0.828086
Epoch 62
Validation binary_cross_entropy = 0.719535
Epoch 63
Validation binary_cross_entropy = 0.624191
Epoch 64
Loss = 1.0174e-01, PNorm = 89.1870, GNorm = 0.6223, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.597761
Epoch 65
Validation binary_cross_entropy = 0.593361
Epoch 66
Validation binary_cross_entropy = 0.575373
Epoch 67
Validation binary_cross_entropy = 0.592017
Epoch 68
Validation binary_cross_entropy = 0.634342
Epoch 69
Loss = 1.1195e-01, PNorm = 89.4995, GNorm = 0.2274, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.666030
Epoch 70
Validation binary_cross_entropy = 0.713648
Epoch 71
Validation binary_cross_entropy = 0.767533
Epoch 72
Validation binary_cross_entropy = 0.815814
Epoch 73
Validation binary_cross_entropy = 1.001131
Epoch 74
Loss = 8.0651e-02, PNorm = 89.7619, GNorm = 2.0209, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.836189
Epoch 75
Validation binary_cross_entropy = 0.895823
Epoch 76
Validation binary_cross_entropy = 0.804025
Epoch 77
Validation binary_cross_entropy = 0.834063
Epoch 78
Validation binary_cross_entropy = 0.864578
Epoch 79
Loss = 1.1914e-01, PNorm = 89.9963, GNorm = 0.8963, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.782955
Epoch 80
Validation binary_cross_entropy = 0.785033
Epoch 81
Validation binary_cross_entropy = 0.851355
Epoch 82
Validation binary_cross_entropy = 0.864514
Epoch 83
Validation binary_cross_entropy = 0.831925
Epoch 84
Loss = 5.7479e-02, PNorm = 90.2344, GNorm = 1.4340, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.845686
Epoch 85
Validation binary_cross_entropy = 0.871606
Epoch 86
Validation binary_cross_entropy = 0.777929
Epoch 87
Validation binary_cross_entropy = 0.775737
Epoch 88
Validation binary_cross_entropy = 0.776923
Epoch 89
Loss = 9.6913e-02, PNorm = 90.4891, GNorm = 0.2878, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.783942
Epoch 90
Validation binary_cross_entropy = 0.790082
Epoch 91
Validation binary_cross_entropy = 0.855275
Epoch 92
Validation binary_cross_entropy = 0.956845
Epoch 93
Validation binary_cross_entropy = 1.016732
Epoch 94
Loss = 2.3819e-02, PNorm = 90.6960, GNorm = 1.2574, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.071643
Epoch 95
Validation binary_cross_entropy = 1.122338
Epoch 96
Validation binary_cross_entropy = 1.131564
Epoch 97
Validation binary_cross_entropy = 1.138877
Epoch 98
Validation binary_cross_entropy = 1.181096
Epoch 99
Loss = 1.4650e-02, PNorm = 90.8505, GNorm = 1.1654, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.222480
Epoch 100
Validation binary_cross_entropy = 1.195462
Epoch 101
Validation binary_cross_entropy = 1.385630
Epoch 102
Validation binary_cross_entropy = 1.199795
Epoch 103
Validation binary_cross_entropy = 1.153128
Epoch 104
Loss = 6.2525e-02, PNorm = 90.9625, GNorm = 0.3260, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 1.171016
Epoch 105
Validation binary_cross_entropy = 1.179882
Epoch 106
Validation binary_cross_entropy = 1.224679
Epoch 107
Validation binary_cross_entropy = 1.183838
Epoch 108
Validation binary_cross_entropy = 1.031318
Epoch 109
Loss = 1.0551e-01, PNorm = 91.0782, GNorm = 0.2680, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.959916
Epoch 110
Validation binary_cross_entropy = 1.025256
Epoch 111
Validation binary_cross_entropy = 1.038033
Epoch 112
Validation binary_cross_entropy = 0.992164
Epoch 113
Validation binary_cross_entropy = 0.956157
Epoch 114
Loss = 9.5854e-02, PNorm = 91.3045, GNorm = 0.7142, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 1.045949
Epoch 115
Validation binary_cross_entropy = 1.111921
Epoch 116
Validation binary_cross_entropy = 1.073621
Epoch 117
Validation binary_cross_entropy = 1.063095
Epoch 118
Validation binary_cross_entropy = 1.066196
Epoch 119
Loss = 3.5451e-02, PNorm = 91.5072, GNorm = 0.7331, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 1.100898
Epoch 120
Validation binary_cross_entropy = 1.184314
Epoch 121
Validation binary_cross_entropy = 1.120529
Epoch 122
Validation binary_cross_entropy = 1.099907
Epoch 123
Validation binary_cross_entropy = 1.171963
Epoch 124
Loss = 4.6886e-02, PNorm = 91.6906, GNorm = 0.4260, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 1.261772
Epoch 125
Validation binary_cross_entropy = 1.243169
Epoch 126
Validation binary_cross_entropy = 1.204612
Epoch 127
Validation binary_cross_entropy = 1.207361
Epoch 128
Validation binary_cross_entropy = 1.328402
Epoch 129
Loss = 8.6950e-02, PNorm = 91.8409, GNorm = 0.8773, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 1.273395
Epoch 130
Validation binary_cross_entropy = 1.259673
Epoch 131
Validation binary_cross_entropy = 1.243020
Epoch 132
Validation binary_cross_entropy = 1.278368
Epoch 133
Validation binary_cross_entropy = 1.252073
Epoch 134
Loss = 3.7748e-02, PNorm = 91.9922, GNorm = 1.1185, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 1.258567
Epoch 135
Validation binary_cross_entropy = 1.339677
Epoch 136
Validation binary_cross_entropy = 1.234536
Epoch 137
Validation binary_cross_entropy = 1.227209
Epoch 138
Validation binary_cross_entropy = 1.294928
Epoch 139
Loss = 1.2766e-01, PNorm = 92.1458, GNorm = 0.0279, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 1.254686
Epoch 140
Validation binary_cross_entropy = 1.351642
Epoch 141
Validation binary_cross_entropy = 1.225100
Epoch 142
Validation binary_cross_entropy = 1.126257
Epoch 143
Validation binary_cross_entropy = 1.070708
Epoch 144
Loss = 1.5184e-01, PNorm = 92.2917, GNorm = 1.9006, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.116812
Epoch 145
Validation binary_cross_entropy = 1.245434
Epoch 146
Validation binary_cross_entropy = 1.112468
Epoch 147
Validation binary_cross_entropy = 0.951632
Epoch 148
Validation binary_cross_entropy = 0.890676
Epoch 149
Loss = 2.8588e-02, PNorm = 92.4714, GNorm = 0.7262, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 0.902367
Epoch 150
Validation binary_cross_entropy = 0.899259
Epoch 151
Validation binary_cross_entropy = 0.973943
Epoch 152
Validation binary_cross_entropy = 1.067857
Epoch 153
Validation binary_cross_entropy = 1.139286
Epoch 154
Loss = 6.8757e-02, PNorm = 92.6565, GNorm = 0.6130, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 1.281327
Epoch 155
Validation binary_cross_entropy = 1.354534
Epoch 156
Validation binary_cross_entropy = 1.401403
Epoch 157
Validation binary_cross_entropy = 1.472844
Epoch 158
Validation binary_cross_entropy = 1.534658
Epoch 159
Loss = 1.9543e-02, PNorm = 92.7940, GNorm = 0.3573, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.599616
Epoch 160
Validation binary_cross_entropy = 1.567238
Epoch 161
Validation binary_cross_entropy = 1.515790
Epoch 162
Validation binary_cross_entropy = 1.583230
Epoch 163
Validation binary_cross_entropy = 1.669597
Epoch 164
Loss = 1.3643e-02, PNorm = 92.8990, GNorm = 0.4799, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.760163
Epoch 165
Validation binary_cross_entropy = 1.847674
Epoch 166
Validation binary_cross_entropy = 1.913198
Epoch 167
Validation binary_cross_entropy = 1.942131
Epoch 168
Validation binary_cross_entropy = 1.930933
Epoch 169
Loss = 3.5063e-03, PNorm = 92.9869, GNorm = 0.0540, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.911356
Epoch 170
Validation binary_cross_entropy = 1.893111
Epoch 171
Validation binary_cross_entropy = 1.880615
Epoch 172
Validation binary_cross_entropy = 1.877248
Epoch 173
Validation binary_cross_entropy = 1.885146
Epoch 174
Loss = 1.2284e-02, PNorm = 93.0428, GNorm = 0.9490, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.909870
Epoch 175
Validation binary_cross_entropy = 1.963641
Epoch 176
Validation binary_cross_entropy = 2.019252
Epoch 177
Validation binary_cross_entropy = 2.056555
Epoch 178
Validation binary_cross_entropy = 2.057727
Epoch 179
Loss = 2.1654e-03, PNorm = 93.0921, GNorm = 0.0947, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 2.049124
Epoch 180
Validation binary_cross_entropy = 2.038036
Epoch 181
Validation binary_cross_entropy = 2.029532
Epoch 182
Validation binary_cross_entropy = 2.023898
Epoch 183
Validation binary_cross_entropy = 2.020902
Epoch 184
Loss = 1.8856e-04, PNorm = 93.1253, GNorm = 0.0032, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 2.025896
Epoch 185
Validation binary_cross_entropy = 2.039008
Epoch 186
Validation binary_cross_entropy = 2.046992
Epoch 187
Validation binary_cross_entropy = 2.053528
Epoch 188
Validation binary_cross_entropy = 2.060779
Epoch 189
Loss = 1.1430e-04, PNorm = 93.1419, GNorm = 0.0015, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 2.067389
Epoch 190
Validation binary_cross_entropy = 2.073098
Epoch 191
Validation binary_cross_entropy = 2.231442
Epoch 192
Validation binary_cross_entropy = 2.158569
Epoch 193
Validation binary_cross_entropy = 1.982096
Epoch 194
Loss = 1.0577e-03, PNorm = 93.1773, GNorm = 0.0239, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.843884
Epoch 195
Validation binary_cross_entropy = 1.737398
Epoch 196
Validation binary_cross_entropy = 1.739701
Epoch 197
Validation binary_cross_entropy = 1.797139
Epoch 198
Validation binary_cross_entropy = 1.751545
Epoch 199
Loss = 6.2841e-03, PNorm = 93.2615, GNorm = 0.8804, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.763867
Epoch 200
Validation binary_cross_entropy = 1.626349
Epoch 201
Validation binary_cross_entropy = 1.564620
Epoch 202
Validation binary_cross_entropy = 1.549126
Epoch 203
Validation binary_cross_entropy = 1.637134
Epoch 204
Loss = 1.1788e-02, PNorm = 93.3691, GNorm = 0.7767, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.704128
Epoch 205
Validation binary_cross_entropy = 1.557925
Epoch 206
Validation binary_cross_entropy = 1.474535
Epoch 207
Validation binary_cross_entropy = 1.426585
Epoch 208
Validation binary_cross_entropy = 1.413236
Epoch 209
Loss = 3.4140e-02, PNorm = 93.4745, GNorm = 0.0660, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.461435
Epoch 210
Validation binary_cross_entropy = 1.522847
Epoch 211
Validation binary_cross_entropy = 1.523422
Epoch 212
Validation binary_cross_entropy = 1.491138
Epoch 213
Validation binary_cross_entropy = 1.481150
Epoch 214
Loss = 1.3642e-02, PNorm = 93.5527, GNorm = 0.1308, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.619033
Epoch 215
Validation binary_cross_entropy = 1.713525
Epoch 216
Validation binary_cross_entropy = 1.637300
Epoch 217
Validation binary_cross_entropy = 1.605750
Epoch 218
Validation binary_cross_entropy = 1.584889
Epoch 219
Loss = 6.0503e-04, PNorm = 93.6294, GNorm = 0.0253, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.570779
Epoch 220
Validation binary_cross_entropy = 1.597958
Epoch 221
Validation binary_cross_entropy = 1.692374
Epoch 222
Validation binary_cross_entropy = 1.699161
Epoch 223
Validation binary_cross_entropy = 1.629425
Epoch 224
Loss = 5.1577e-04, PNorm = 93.6864, GNorm = 0.0216, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.597150
Epoch 225
Validation binary_cross_entropy = 1.586068
Epoch 226
Validation binary_cross_entropy = 1.588004
Epoch 227
Validation binary_cross_entropy = 1.612026
Epoch 228
Validation binary_cross_entropy = 1.661740
Epoch 229
Loss = 3.9729e-02, PNorm = 93.7454, GNorm = 1.1788, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.772962
Epoch 230
Validation binary_cross_entropy = 1.965559
Epoch 231
Validation binary_cross_entropy = 1.901253
Epoch 232
Validation binary_cross_entropy = 1.722516
Epoch 233
Validation binary_cross_entropy = 1.627024
Epoch 234
Loss = 1.4904e-02, PNorm = 93.8151, GNorm = 0.2433, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.729407
Epoch 235
Validation binary_cross_entropy = 1.817207
Epoch 236
Validation binary_cross_entropy = 1.912712
Epoch 237
Validation binary_cross_entropy = 2.027294
Epoch 238
Validation binary_cross_entropy = 1.965074
Epoch 239
Loss = 4.5787e-03, PNorm = 93.8858, GNorm = 0.4137, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.708817
Epoch 240
Validation binary_cross_entropy = 1.584969
Epoch 241
Validation binary_cross_entropy = 1.555572
Epoch 242
Validation binary_cross_entropy = 1.594565
Epoch 243
Validation binary_cross_entropy = 1.558144
Epoch 244
Loss = 1.6505e-02, PNorm = 93.9876, GNorm = 0.7065, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.558506
Epoch 245
Validation binary_cross_entropy = 1.590279
Epoch 246
Validation binary_cross_entropy = 1.616256
Epoch 247
Validation binary_cross_entropy = 1.584085
Epoch 248
Validation binary_cross_entropy = 1.501107
Epoch 249
Loss = 1.0702e-03, PNorm = 94.0851, GNorm = 0.0536, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.440734
Epoch 250
Validation binary_cross_entropy = 1.452674
Epoch 251
Validation binary_cross_entropy = 1.451553
Epoch 252
Validation binary_cross_entropy = 1.420306
Epoch 253
Validation binary_cross_entropy = 1.384213
Epoch 254
Loss = 1.3531e-03, PNorm = 94.1500, GNorm = 0.0552, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.362028
Epoch 255
Validation binary_cross_entropy = 1.358311
Epoch 256
Validation binary_cross_entropy = 1.368278
Epoch 257
Validation binary_cross_entropy = 1.382596
Epoch 258
Validation binary_cross_entropy = 1.405315
Epoch 259
Loss = 1.6594e-03, PNorm = 94.1894, GNorm = 0.1305, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.440966
Epoch 260
Validation binary_cross_entropy = 1.478874
Epoch 261
Validation binary_cross_entropy = 1.503080
Epoch 262
Validation binary_cross_entropy = 1.514099
Epoch 263
Validation binary_cross_entropy = 1.508977
Epoch 264
Loss = 1.3456e-03, PNorm = 94.2255, GNorm = 0.0494, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.488018
Epoch 265
Validation binary_cross_entropy = 1.444177
Epoch 266
Validation binary_cross_entropy = 1.423267
Epoch 267
Validation binary_cross_entropy = 1.442849
Epoch 268
Validation binary_cross_entropy = 1.477410
Epoch 269
Loss = 1.8905e-04, PNorm = 94.2481, GNorm = 0.0112, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.508722
Epoch 270
Validation binary_cross_entropy = 1.535587
Epoch 271
Validation binary_cross_entropy = 1.558564
Epoch 272
Validation binary_cross_entropy = 1.577402
Epoch 273
Validation binary_cross_entropy = 1.590518
Epoch 274
Loss = 1.4042e-04, PNorm = 94.2599, GNorm = 0.0032, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.598950
Epoch 275
Validation binary_cross_entropy = 1.616702
Epoch 276
Validation binary_cross_entropy = 1.629677
Epoch 277
Validation binary_cross_entropy = 1.640225
Epoch 278
Validation binary_cross_entropy = 1.649399
Epoch 279
Loss = 1.9834e-04, PNorm = 94.2694, GNorm = 0.0054, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.656623
Epoch 280
Validation binary_cross_entropy = 1.662606
Epoch 281
Validation binary_cross_entropy = 1.668315
Epoch 282
Validation binary_cross_entropy = 1.673115
Epoch 283
Validation binary_cross_entropy = 1.676975
Epoch 284
Loss = 1.6214e-04, PNorm = 94.2764, GNorm = 0.0038, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.680461
Epoch 285
Validation binary_cross_entropy = 1.683476
Epoch 286
Validation binary_cross_entropy = 1.686134
Epoch 287
Validation binary_cross_entropy = 1.688747
Epoch 288
Validation binary_cross_entropy = 1.691290
Epoch 289
Loss = 8.9084e-05, PNorm = 94.2816, GNorm = 0.0017, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.693371
Epoch 290
Validation binary_cross_entropy = 1.695768
Epoch 291
Validation binary_cross_entropy = 1.698085
Epoch 292
Validation binary_cross_entropy = 1.700489
Epoch 293
Validation binary_cross_entropy = 1.704010
Epoch 294
Loss = 6.8947e-05, PNorm = 94.2857, GNorm = 0.0012, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.707308
Epoch 295
Validation binary_cross_entropy = 1.710191
Epoch 296
Validation binary_cross_entropy = 1.712732
Epoch 297
Validation binary_cross_entropy = 1.715798
Epoch 298
Validation binary_cross_entropy = 1.718894
Epoch 299
Loss = 5.4909e-05, PNorm = 94.2892, GNorm = 0.0014, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.721628
Model 0 best validation binary_cross_entropy = 0.275943 on epoch 30
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.226705
Ensemble test binary_cross_entropy = 0.226705
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1700, bias=False)
        (W_h): Linear(in_features=1700, out_features=1700, bias=False)
        (W_o): Linear(in_features=1833, out_features=1700, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=1700, out_features=1700, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=1700, out_features=1700, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.0, inplace=False)
    (7): Linear(in_features=1700, out_features=1, bias=True)
  )
)
Number of parameters = 12,042,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.641183
Epoch 1
Validation binary_cross_entropy = 0.647575
Epoch 2
Validation binary_cross_entropy = 0.728634
Epoch 3
Validation binary_cross_entropy = 0.681680
Epoch 4
Loss = 6.7054e-01, PNorm = 84.5187, GNorm = 0.4881, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.596093
Epoch 5
Validation binary_cross_entropy = 0.629473
Epoch 6
Validation binary_cross_entropy = 0.785578
Epoch 7
Validation binary_cross_entropy = 0.424659
Epoch 8
Validation binary_cross_entropy = 1.009228
Epoch 9
Loss = 6.5068e-01, PNorm = 84.7952, GNorm = 0.3838, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.635760
Epoch 10
Validation binary_cross_entropy = 0.621022
Epoch 11
Validation binary_cross_entropy = 0.526018
Epoch 12
Validation binary_cross_entropy = 0.618732
Epoch 13
Validation binary_cross_entropy = 0.461188
Epoch 14
Loss = 5.8736e-01, PNorm = 85.2194, GNorm = 0.2275, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.390717
Epoch 15
Validation binary_cross_entropy = 0.826910
Epoch 16
Validation binary_cross_entropy = 0.300006
Epoch 17
Validation binary_cross_entropy = 0.567477
Epoch 18
Validation binary_cross_entropy = 0.454383
Epoch 19
Loss = 4.9806e-01, PNorm = 85.6160, GNorm = 0.7347, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.333170
Epoch 20
Validation binary_cross_entropy = 0.696112
Epoch 21
Validation binary_cross_entropy = 0.351933
Epoch 22
Validation binary_cross_entropy = 0.350642
Epoch 23
Validation binary_cross_entropy = 0.528609
Epoch 24
Loss = 4.4663e-01, PNorm = 85.9778, GNorm = 0.5266, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.350019
Epoch 25
Validation binary_cross_entropy = 0.363042
Epoch 26
Validation binary_cross_entropy = 0.447443
Epoch 27
Validation binary_cross_entropy = 0.328907
Epoch 28
Validation binary_cross_entropy = 0.329512
Epoch 29
Loss = 3.9674e-01, PNorm = 86.3266, GNorm = 1.2956, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.497939
Epoch 30
Validation binary_cross_entropy = 0.466213
Epoch 31
Validation binary_cross_entropy = 0.344446
Epoch 32
Validation binary_cross_entropy = 0.347063
Epoch 33
Validation binary_cross_entropy = 0.364991
Epoch 34
Loss = 2.7458e-01, PNorm = 86.6883, GNorm = 0.9282, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.383142
Epoch 35
Validation binary_cross_entropy = 0.355872
Epoch 36
Validation binary_cross_entropy = 0.414082
Epoch 37
Validation binary_cross_entropy = 0.435168
Epoch 38
Validation binary_cross_entropy = 0.393544
Epoch 39
Loss = 2.6142e-01, PNorm = 87.0515, GNorm = 4.4921, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.629158
Epoch 40
Validation binary_cross_entropy = 0.374097
Epoch 41
Validation binary_cross_entropy = 0.416568
Epoch 42
Validation binary_cross_entropy = 0.521592
Epoch 43
Validation binary_cross_entropy = 0.289703
Epoch 44
Loss = 4.1663e-01, PNorm = 87.4208, GNorm = 0.8213, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.612080
Epoch 45
Validation binary_cross_entropy = 0.454225
Epoch 46
Validation binary_cross_entropy = 0.327370
Epoch 47
Validation binary_cross_entropy = 0.355061
Epoch 48
Validation binary_cross_entropy = 0.392554
Epoch 49
Loss = 2.8321e-01, PNorm = 87.8151, GNorm = 0.5195, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.376380
Epoch 50
Validation binary_cross_entropy = 0.369415
Epoch 51
Validation binary_cross_entropy = 0.409410
Epoch 52
Validation binary_cross_entropy = 0.468087
Epoch 53
Validation binary_cross_entropy = 0.495797
Epoch 54
Loss = 1.5904e-01, PNorm = 88.2204, GNorm = 0.8661, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.656798
Epoch 55
Validation binary_cross_entropy = 0.526604
Epoch 56
Validation binary_cross_entropy = 0.498223
Epoch 57
Validation binary_cross_entropy = 0.549402
Epoch 58
Validation binary_cross_entropy = 0.537948
Epoch 59
Loss = 2.3623e-01, PNorm = 88.5700, GNorm = 1.0075, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.519781
Epoch 60
Validation binary_cross_entropy = 0.500206
Epoch 61
Validation binary_cross_entropy = 0.584989
Epoch 62
Validation binary_cross_entropy = 0.620011
Epoch 63
Validation binary_cross_entropy = 0.557535
Epoch 64
Loss = 1.1484e-01, PNorm = 88.8856, GNorm = 0.4249, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.509489
Epoch 65
Validation binary_cross_entropy = 0.531656
Epoch 66
Validation binary_cross_entropy = 0.559817
Epoch 67
Validation binary_cross_entropy = 0.553980
Epoch 68
Validation binary_cross_entropy = 0.592066
Epoch 69
Loss = 1.3838e-01, PNorm = 89.2364, GNorm = 0.8272, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.605496
Epoch 70
Validation binary_cross_entropy = 0.683543
Epoch 71
Validation binary_cross_entropy = 0.725126
Epoch 72
Validation binary_cross_entropy = 0.613088
Epoch 73
Validation binary_cross_entropy = 0.591954
Epoch 74
Loss = 1.0180e-01, PNorm = 89.5375, GNorm = 0.3977, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.581093
Epoch 75
Validation binary_cross_entropy = 0.597216
Epoch 76
Validation binary_cross_entropy = 0.607624
Epoch 77
Validation binary_cross_entropy = 0.651156
Epoch 78
Validation binary_cross_entropy = 0.624294
Epoch 79
Loss = 9.0045e-02, PNorm = 89.8089, GNorm = 0.8265, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.615631
Epoch 80
Validation binary_cross_entropy = 0.617300
Epoch 81
Validation binary_cross_entropy = 0.668937
Epoch 82
Validation binary_cross_entropy = 0.659877
Epoch 83
Validation binary_cross_entropy = 0.727929
Epoch 84
Loss = 9.8412e-02, PNorm = 90.0679, GNorm = 1.0902, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.706246
Epoch 85
Validation binary_cross_entropy = 0.754516
Epoch 86
Validation binary_cross_entropy = 0.730797
Epoch 87
Validation binary_cross_entropy = 0.742162
Epoch 88
Validation binary_cross_entropy = 0.711533
Epoch 89
Loss = 1.3385e-01, PNorm = 90.2898, GNorm = 4.0907, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.720093
Epoch 90
Validation binary_cross_entropy = 0.695442
Epoch 91
Validation binary_cross_entropy = 0.685073
Epoch 92
Validation binary_cross_entropy = 0.687262
Epoch 93
Validation binary_cross_entropy = 0.685099
Epoch 94
Loss = 1.5407e-01, PNorm = 90.4877, GNorm = 1.1479, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.686505
Epoch 95
Validation binary_cross_entropy = 0.706531
Epoch 96
Validation binary_cross_entropy = 0.760349
Epoch 97
Validation binary_cross_entropy = 0.757509
Epoch 98
Validation binary_cross_entropy = 0.793976
Epoch 99
Loss = 6.7368e-02, PNorm = 90.6707, GNorm = 0.9186, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.823483
Epoch 100
Validation binary_cross_entropy = 0.845140
Epoch 101
Validation binary_cross_entropy = 0.891570
Epoch 102
Validation binary_cross_entropy = 0.919371
Epoch 103
Validation binary_cross_entropy = 0.945394
Epoch 104
Loss = 8.7554e-02, PNorm = 90.8489, GNorm = 1.8509, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.925950
Epoch 105
Validation binary_cross_entropy = 0.928192
Epoch 106
Validation binary_cross_entropy = 0.908409
Epoch 107
Validation binary_cross_entropy = 0.894165
Epoch 108
Validation binary_cross_entropy = 0.838414
Epoch 109
Loss = 5.4305e-02, PNorm = 90.9934, GNorm = 0.6826, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.794629
Epoch 110
Validation binary_cross_entropy = 0.779352
Epoch 111
Validation binary_cross_entropy = 0.789597
Epoch 112
Validation binary_cross_entropy = 0.826724
Epoch 113
Validation binary_cross_entropy = 0.876499
Epoch 114
Loss = 4.5981e-02, PNorm = 91.1427, GNorm = 0.5707, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.920974
Epoch 115
Validation binary_cross_entropy = 0.957952
Epoch 116
Validation binary_cross_entropy = 0.980419
Epoch 117
Validation binary_cross_entropy = 0.995263
Epoch 118
Validation binary_cross_entropy = 1.015574
Epoch 119
Loss = 7.2501e-02, PNorm = 91.2750, GNorm = 0.7203, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 1.007421
Epoch 120
Validation binary_cross_entropy = 1.025405
Epoch 121
Validation binary_cross_entropy = 1.077619
Epoch 122
Validation binary_cross_entropy = 1.120904
Epoch 123
Validation binary_cross_entropy = 1.158834
Epoch 124
Loss = 2.0364e-02, PNorm = 91.3912, GNorm = 0.1271, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 1.189351
Epoch 125
Validation binary_cross_entropy = 1.225571
Epoch 126
Validation binary_cross_entropy = 1.233940
Epoch 127
Validation binary_cross_entropy = 1.241047
Epoch 128
Validation binary_cross_entropy = 1.254588
Epoch 129
Loss = 1.2071e-02, PNorm = 91.4863, GNorm = 0.4748, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 1.266263
Epoch 130
Validation binary_cross_entropy = 1.281698
Epoch 131
Validation binary_cross_entropy = 1.302684
Epoch 132
Validation binary_cross_entropy = 1.337993
Epoch 133
Validation binary_cross_entropy = 1.381616
Epoch 134
Loss = 3.5097e-02, PNorm = 91.5755, GNorm = 0.6913, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 1.404682
Epoch 135
Validation binary_cross_entropy = 1.426464
Epoch 136
Validation binary_cross_entropy = 1.410445
Epoch 137
Validation binary_cross_entropy = 1.427373
Epoch 138
Validation binary_cross_entropy = 1.445584
Epoch 139
Loss = 5.2964e-03, PNorm = 91.6374, GNorm = 0.0177, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 1.446224
Epoch 140
Validation binary_cross_entropy = 1.487205
Epoch 141
Validation binary_cross_entropy = 1.512627
Epoch 142
Validation binary_cross_entropy = 1.525908
Epoch 143
Validation binary_cross_entropy = 1.562653
Epoch 144
Loss = 4.2578e-03, PNorm = 91.7026, GNorm = 0.2036, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.579649
Epoch 145
Validation binary_cross_entropy = 1.547806
Epoch 146
Validation binary_cross_entropy = 1.576731
Epoch 147
Validation binary_cross_entropy = 1.585599
Epoch 148
Validation binary_cross_entropy = 1.605762
Epoch 149
Loss = 8.6296e-03, PNorm = 91.7567, GNorm = 0.5083, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.619924
Epoch 150
Validation binary_cross_entropy = 1.618657
Epoch 151
Validation binary_cross_entropy = 1.621526
Epoch 152
Validation binary_cross_entropy = 1.633373
Epoch 153
Validation binary_cross_entropy = 1.651628
Epoch 154
Loss = 5.4600e-04, PNorm = 91.7965, GNorm = 0.0077, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 1.672577
Epoch 155
Validation binary_cross_entropy = 1.691825
Epoch 156
Validation binary_cross_entropy = 1.707541
Epoch 157
Validation binary_cross_entropy = 1.720650
Epoch 158
Validation binary_cross_entropy = 1.731684
Epoch 159
Loss = 1.9573e-03, PNorm = 91.8220, GNorm = 0.0025, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.740426
Epoch 160
Validation binary_cross_entropy = 1.745733
Epoch 161
Validation binary_cross_entropy = 1.753259
Epoch 162
Validation binary_cross_entropy = 1.761355
Epoch 163
Validation binary_cross_entropy = 1.763346
Epoch 164
Loss = 7.2617e-04, PNorm = 91.8443, GNorm = 0.0487, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.760506
Epoch 165
Validation binary_cross_entropy = 1.765762
Epoch 166
Validation binary_cross_entropy = 1.774815
Epoch 167
Validation binary_cross_entropy = 1.787290
Epoch 168
Validation binary_cross_entropy = 1.796800
Epoch 169
Loss = 3.3034e-04, PNorm = 91.8629, GNorm = 0.0160, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.804797
Epoch 170
Validation binary_cross_entropy = 1.811960
Epoch 171
Validation binary_cross_entropy = 1.819047
Epoch 172
Validation binary_cross_entropy = 1.825591
Epoch 173
Validation binary_cross_entropy = 1.831211
Epoch 174
Loss = 9.1033e-05, PNorm = 91.8740, GNorm = 0.0031, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.836820
Epoch 175
Validation binary_cross_entropy = 1.842275
Epoch 176
Validation binary_cross_entropy = 1.848091
Epoch 177
Validation binary_cross_entropy = 1.852399
Epoch 178
Validation binary_cross_entropy = 1.856892
Epoch 179
Loss = 1.2207e-04, PNorm = 91.8817, GNorm = 0.0053, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.862707
Epoch 180
Validation binary_cross_entropy = 1.866532
Epoch 181
Validation binary_cross_entropy = 1.870134
Epoch 182
Validation binary_cross_entropy = 1.874646
Epoch 183
Validation binary_cross_entropy = 1.878814
Epoch 184
Loss = 9.7515e-05, PNorm = 91.8877, GNorm = 0.0022, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.882706
Epoch 185
Validation binary_cross_entropy = 1.886498
Epoch 186
Validation binary_cross_entropy = 1.890133
Epoch 187
Validation binary_cross_entropy = 1.893581
Epoch 188
Validation binary_cross_entropy = 1.896833
Epoch 189
Loss = 8.2320e-05, PNorm = 91.8929, GNorm = 0.0024, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.899925
Epoch 190
Validation binary_cross_entropy = 1.902834
Epoch 191
Validation binary_cross_entropy = 1.908263
Epoch 192
Validation binary_cross_entropy = 1.910926
Epoch 193
Validation binary_cross_entropy = 1.913378
Epoch 194
Loss = 7.5233e-05, PNorm = 91.8968, GNorm = 0.0025, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.915709
Epoch 195
Validation binary_cross_entropy = 1.917965
Epoch 196
Validation binary_cross_entropy = 1.919988
Epoch 197
Validation binary_cross_entropy = 1.921981
Epoch 198
Validation binary_cross_entropy = 1.923866
Epoch 199
Loss = 5.8357e-05, PNorm = 91.8999, GNorm = 0.0021, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.925654
Epoch 200
Validation binary_cross_entropy = 1.927275
Epoch 201
Validation binary_cross_entropy = 1.933311
Epoch 202
Validation binary_cross_entropy = 1.935041
Epoch 203
Validation binary_cross_entropy = 1.937010
Epoch 204
Loss = 4.4607e-05, PNorm = 91.9024, GNorm = 0.0012, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.938749
Epoch 205
Validation binary_cross_entropy = 1.940403
Epoch 206
Validation binary_cross_entropy = 1.942115
Epoch 207
Validation binary_cross_entropy = 1.943916
Epoch 208
Validation binary_cross_entropy = 1.945662
Epoch 209
Loss = 8.7320e-05, PNorm = 91.9052, GNorm = 0.0015, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.947410
Epoch 210
Validation binary_cross_entropy = 1.949127
Epoch 211
Validation binary_cross_entropy = 1.950639
Epoch 212
Validation binary_cross_entropy = 1.952058
Epoch 213
Validation binary_cross_entropy = 1.953413
Epoch 214
Loss = 5.7346e-05, PNorm = 91.9079, GNorm = 0.0013, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.954751
Epoch 215
Validation binary_cross_entropy = 1.956021
Epoch 216
Validation binary_cross_entropy = 1.957217
Epoch 217
Validation binary_cross_entropy = 1.958598
Epoch 218
Validation binary_cross_entropy = 1.959880
Epoch 219
Loss = 4.9276e-05, PNorm = 91.9100, GNorm = 0.0011, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.961245
Epoch 220
Validation binary_cross_entropy = 1.962865
Epoch 221
Validation binary_cross_entropy = 1.964319
Epoch 222
Validation binary_cross_entropy = 1.966295
Epoch 223
Validation binary_cross_entropy = 1.968131
Epoch 224
Loss = 5.3783e-05, PNorm = 91.9124, GNorm = 0.0025, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.969803
Epoch 225
Validation binary_cross_entropy = 1.971318
Epoch 226
Validation binary_cross_entropy = 1.972677
Epoch 227
Validation binary_cross_entropy = 1.973954
Epoch 228
Validation binary_cross_entropy = 1.975427
Epoch 229
Loss = 4.1632e-05, PNorm = 91.9149, GNorm = 0.0011, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.977002
Epoch 230
Validation binary_cross_entropy = 1.978529
Epoch 231
Validation binary_cross_entropy = 1.979882
Epoch 232
Validation binary_cross_entropy = 1.981044
Epoch 233
Validation binary_cross_entropy = 1.982348
Epoch 234
Loss = 4.2422e-05, PNorm = 91.9170, GNorm = 0.0029, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.983512
Epoch 235
Validation binary_cross_entropy = 1.984986
Epoch 236
Validation binary_cross_entropy = 1.986730
Epoch 237
Validation binary_cross_entropy = 1.988323
Epoch 238
Validation binary_cross_entropy = 1.989784
Epoch 239
Loss = 3.0067e-05, PNorm = 91.9193, GNorm = 0.0009, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.991068
Epoch 240
Validation binary_cross_entropy = 1.992264
Epoch 241
Validation binary_cross_entropy = 1.993528
Epoch 242
Validation binary_cross_entropy = 1.994730
Epoch 243
Validation binary_cross_entropy = 1.995942
Epoch 244
Loss = 3.6950e-05, PNorm = 91.9211, GNorm = 0.0006, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.997085
Epoch 245
Validation binary_cross_entropy = 1.998331
Epoch 246
Validation binary_cross_entropy = 1.999527
Epoch 247
Validation binary_cross_entropy = 2.000832
Epoch 248
Validation binary_cross_entropy = 2.537394
Epoch 249
Loss = 5.4797e-05, PNorm = 91.9231, GNorm = 0.0016, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 2.538831
Epoch 250
Validation binary_cross_entropy = 2.540491
Epoch 251
Validation binary_cross_entropy = 2.542000
Epoch 252
Validation binary_cross_entropy = 2.543464
Epoch 253
Validation binary_cross_entropy = 2.544933
Epoch 254
Loss = 4.3413e-05, PNorm = 91.9252, GNorm = 0.0019, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 2.546400
Epoch 255
Validation binary_cross_entropy = 2.548056
Epoch 256
Validation binary_cross_entropy = 2.549897
Epoch 257
Validation binary_cross_entropy = 2.551772
Epoch 258
Validation binary_cross_entropy = 2.553399
Epoch 259
Loss = 3.6261e-05, PNorm = 91.9277, GNorm = 0.0006, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 2.554989
Epoch 260
Validation binary_cross_entropy = 2.556607
Epoch 261
Validation binary_cross_entropy = 2.558152
Epoch 262
Validation binary_cross_entropy = 2.559521
Epoch 263
Validation binary_cross_entropy = 2.560725
Epoch 264
Loss = 2.9013e-05, PNorm = 91.9298, GNorm = 0.0007, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 2.561899
Epoch 265
Validation binary_cross_entropy = 2.562896
Epoch 266
Validation binary_cross_entropy = 2.563811
Epoch 267
Validation binary_cross_entropy = 2.564571
Epoch 268
Validation binary_cross_entropy = 2.565331
Epoch 269
Loss = 2.4906e-05, PNorm = 91.9315, GNorm = 0.0008, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 2.566131
Epoch 270
Validation binary_cross_entropy = 2.567010
Epoch 271
Validation binary_cross_entropy = 2.568123
Epoch 272
Validation binary_cross_entropy = 2.569188
Epoch 273
Validation binary_cross_entropy = 2.570141
Epoch 274
Loss = 2.1329e-05, PNorm = 91.9335, GNorm = 0.0006, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 2.570980
Epoch 275
Validation binary_cross_entropy = 2.571963
Epoch 276
Validation binary_cross_entropy = 2.572917
Epoch 277
Validation binary_cross_entropy = 2.573984
Epoch 278
Validation binary_cross_entropy = 2.575059
Epoch 279
Loss = 2.6311e-05, PNorm = 91.9353, GNorm = 0.0014, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 2.576096
Epoch 280
Validation binary_cross_entropy = 2.577060
Epoch 281
Validation binary_cross_entropy = 2.578236
Epoch 282
Validation binary_cross_entropy = 2.579463
Epoch 283
Validation binary_cross_entropy = 2.580694
Epoch 284
Loss = 2.5058e-05, PNorm = 91.9373, GNorm = 0.0012, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 2.581878
Epoch 285
Validation binary_cross_entropy = 2.583181
Epoch 286
Validation binary_cross_entropy = 2.584411
Epoch 287
Validation binary_cross_entropy = 2.585685
Epoch 288
Validation binary_cross_entropy = 2.586756
Epoch 289
Loss = 2.6458e-05, PNorm = 91.9392, GNorm = 0.0015, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 2.587812
Epoch 290
Validation binary_cross_entropy = 2.588736
Epoch 291
Validation binary_cross_entropy = 2.589525
Epoch 292
Validation binary_cross_entropy = 2.590451
Epoch 293
Validation binary_cross_entropy = 2.591281
Epoch 294
Loss = 2.0857e-05, PNorm = 91.9406, GNorm = 0.0007, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 2.592068
Epoch 295
Validation binary_cross_entropy = 2.593137
Epoch 296
Validation binary_cross_entropy = 2.594156
Epoch 297
Validation binary_cross_entropy = 2.595199
Epoch 298
Validation binary_cross_entropy = 2.596161
Epoch 299
Loss = 2.3992e-05, PNorm = 91.9422, GNorm = 0.0006, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 2.597162
Model 0 best validation binary_cross_entropy = 0.289703 on epoch 43
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.167744
Ensemble test binary_cross_entropy = 0.167744
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1700, bias=False)
        (W_h): Linear(in_features=1700, out_features=1700, bias=False)
        (W_o): Linear(in_features=1833, out_features=1700, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=1700, out_features=1700, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=1700, out_features=1700, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.0, inplace=False)
    (7): Linear(in_features=1700, out_features=1, bias=True)
  )
)
Number of parameters = 12,042,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.623566
Epoch 1
Validation binary_cross_entropy = 0.693540
Epoch 2
Validation binary_cross_entropy = 0.657080
Epoch 3
Validation binary_cross_entropy = 0.703867
Epoch 4
Loss = 6.4457e-01, PNorm = 84.5245, GNorm = 0.3988, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.670991
Epoch 5
Validation binary_cross_entropy = 0.593189
Epoch 6
Validation binary_cross_entropy = 2.267211
Epoch 7
Validation binary_cross_entropy = 0.512595
Epoch 8
Validation binary_cross_entropy = 0.428743
Epoch 9
Loss = 6.6973e-01, PNorm = 84.7941, GNorm = 0.6394, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.551723
Epoch 10
Validation binary_cross_entropy = 0.627816
Epoch 11
Validation binary_cross_entropy = 0.636538
Epoch 12
Validation binary_cross_entropy = 0.457067
Epoch 13
Validation binary_cross_entropy = 0.448248
Epoch 14
Loss = 5.8694e-01, PNorm = 85.2158, GNorm = 0.6806, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.810521
Epoch 15
Validation binary_cross_entropy = 0.266132
Epoch 16
Validation binary_cross_entropy = 0.693464
Epoch 17
Validation binary_cross_entropy = 0.425570
Epoch 18
Validation binary_cross_entropy = 0.294482
Epoch 19
Loss = 4.8830e-01, PNorm = 85.6088, GNorm = 0.4355, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.599553
Epoch 20
Validation binary_cross_entropy = 0.536412
Epoch 21
Validation binary_cross_entropy = 0.362930
Epoch 22
Validation binary_cross_entropy = 0.482695
Epoch 23
Validation binary_cross_entropy = 0.524008
Epoch 24
Loss = 4.4399e-01, PNorm = 85.9968, GNorm = 0.6452, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.225167
Epoch 25
Validation binary_cross_entropy = 0.505561
Epoch 26
Validation binary_cross_entropy = 0.648098
Epoch 27
Validation binary_cross_entropy = 0.327320
Epoch 28
Validation binary_cross_entropy = 0.281543
Epoch 29
Loss = 4.3158e-01, PNorm = 86.3266, GNorm = 0.5199, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.560543
Epoch 30
Validation binary_cross_entropy = 0.571560
Epoch 31
Validation binary_cross_entropy = 0.380861
Epoch 32
Validation binary_cross_entropy = 0.316907
Epoch 33
Validation binary_cross_entropy = 0.382445
Epoch 34
Loss = 2.6686e-01, PNorm = 86.7096, GNorm = 0.4615, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.472441
Epoch 35
Validation binary_cross_entropy = 0.361813
Epoch 36
Validation binary_cross_entropy = 0.341152
Epoch 37
Validation binary_cross_entropy = 0.419272
Epoch 38
Validation binary_cross_entropy = 0.318050
Epoch 39
Loss = 3.9338e-01, PNorm = 87.0535, GNorm = 1.9199, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.300059
Epoch 40
Validation binary_cross_entropy = 0.452718
Epoch 41
Validation binary_cross_entropy = 0.268762
Epoch 42
Validation binary_cross_entropy = 0.351456
Epoch 43
Validation binary_cross_entropy = 0.483996
Epoch 44
Loss = 2.8627e-01, PNorm = 87.4386, GNorm = 1.0446, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.314278
Epoch 45
Validation binary_cross_entropy = 0.411668
Epoch 46
Validation binary_cross_entropy = 0.375676
Epoch 47
Validation binary_cross_entropy = 0.953547
Epoch 48
Validation binary_cross_entropy = 0.477666
Epoch 49
Loss = 2.3452e-01, PNorm = 87.8114, GNorm = 2.9393, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.356944
Epoch 50
Validation binary_cross_entropy = 0.408329
Epoch 51
Validation binary_cross_entropy = 0.335308
Epoch 52
Validation binary_cross_entropy = 0.311763
Epoch 53
Validation binary_cross_entropy = 0.435132
Epoch 54
Loss = 2.5707e-01, PNorm = 88.2018, GNorm = 0.8631, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.421027
Epoch 55
Validation binary_cross_entropy = 0.371874
Epoch 56
Validation binary_cross_entropy = 0.448287
Epoch 57
Validation binary_cross_entropy = 0.525244
Epoch 58
Validation binary_cross_entropy = 0.518640
Epoch 59
Loss = 1.1003e-01, PNorm = 88.6097, GNorm = 1.1757, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.601691
Epoch 60
Validation binary_cross_entropy = 0.648768
Epoch 61
Validation binary_cross_entropy = 0.662271
Epoch 62
Validation binary_cross_entropy = 0.717145
Epoch 63
Validation binary_cross_entropy = 0.613691
Epoch 64
Loss = 8.1469e-02, PNorm = 88.9244, GNorm = 0.5023, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.581359
Epoch 65
Validation binary_cross_entropy = 0.561798
Epoch 66
Validation binary_cross_entropy = 0.591622
Epoch 67
Validation binary_cross_entropy = 0.591689
Epoch 68
Validation binary_cross_entropy = 0.600097
Epoch 69
Loss = 4.3186e-02, PNorm = 89.1751, GNorm = 0.3441, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.754594
Epoch 70
Validation binary_cross_entropy = 0.621966
Epoch 71
Validation binary_cross_entropy = 0.706801
Epoch 72
Validation binary_cross_entropy = 0.664010
Epoch 73
Validation binary_cross_entropy = 0.668967
Epoch 74
Loss = 1.7487e-01, PNorm = 89.3950, GNorm = 0.7192, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.633449
Epoch 75
Validation binary_cross_entropy = 0.630971
Epoch 76
Validation binary_cross_entropy = 0.646483
Epoch 77
Validation binary_cross_entropy = 0.671678
Epoch 78
Validation binary_cross_entropy = 0.707767
Epoch 79
Loss = 2.3466e-02, PNorm = 89.5883, GNorm = 0.4295, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.761171
Epoch 80
Validation binary_cross_entropy = 0.808577
Epoch 81
Validation binary_cross_entropy = 0.860211
Epoch 82
Validation binary_cross_entropy = 0.889943
Epoch 83
Validation binary_cross_entropy = 0.816076
Epoch 84
Loss = 1.0678e-01, PNorm = 89.7524, GNorm = 3.3054, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.776900
Epoch 85
Validation binary_cross_entropy = 0.781291
Epoch 86
Validation binary_cross_entropy = 0.753886
Epoch 87
Validation binary_cross_entropy = 0.748878
Epoch 88
Validation binary_cross_entropy = 0.687123
Epoch 89
Loss = 1.1657e-01, PNorm = 89.9664, GNorm = 1.2616, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.685207
Epoch 90
Validation binary_cross_entropy = 0.703482
Epoch 91
Validation binary_cross_entropy = 0.671061
Epoch 92
Validation binary_cross_entropy = 0.722387
Epoch 93
Validation binary_cross_entropy = 0.705430
Epoch 94
Loss = 1.3192e-01, PNorm = 90.1738, GNorm = 0.7472, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.702913
Epoch 95
Validation binary_cross_entropy = 0.690689
Epoch 96
Validation binary_cross_entropy = 0.746533
Epoch 97
Validation binary_cross_entropy = 0.793653
Epoch 98
Validation binary_cross_entropy = 0.813138
Epoch 99
Loss = 2.2555e-02, PNorm = 90.3656, GNorm = 0.2343, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.844004
Epoch 100
Validation binary_cross_entropy = 0.935872
Epoch 101
Validation binary_cross_entropy = 0.914306
Epoch 102
Validation binary_cross_entropy = 0.949267
Epoch 103
Validation binary_cross_entropy = 0.964347
Epoch 104
Loss = 6.2185e-02, PNorm = 90.5339, GNorm = 1.2488, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.954957
Epoch 105
Validation binary_cross_entropy = 0.950079
Epoch 106
Validation binary_cross_entropy = 0.915008
Epoch 107
Validation binary_cross_entropy = 0.962037
Epoch 108
Validation binary_cross_entropy = 1.052401
Epoch 109
Loss = 1.1804e-01, PNorm = 90.6930, GNorm = 3.9223, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 0.967812
Epoch 110
Validation binary_cross_entropy = 1.012067
Epoch 111
Validation binary_cross_entropy = 0.997515
Epoch 112
Validation binary_cross_entropy = 1.078804
Epoch 113
Validation binary_cross_entropy = 0.917970
Epoch 114
Loss = 3.1111e-02, PNorm = 90.8840, GNorm = 0.3675, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.871032
Epoch 115
Validation binary_cross_entropy = 0.869222
Epoch 116
Validation binary_cross_entropy = 0.891420
Epoch 117
Validation binary_cross_entropy = 0.859250
Epoch 118
Validation binary_cross_entropy = 0.839035
Epoch 119
Loss = 2.2845e-02, PNorm = 91.0617, GNorm = 0.2996, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.887234
Epoch 120
Validation binary_cross_entropy = 0.899805
Epoch 121
Validation binary_cross_entropy = 0.911159
Epoch 122
Validation binary_cross_entropy = 0.921432
Epoch 123
Validation binary_cross_entropy = 0.931063
Epoch 124
Loss = 6.8584e-03, PNorm = 91.1988, GNorm = 0.0368, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.955216
Epoch 125
Validation binary_cross_entropy = 1.014713
Epoch 126
Validation binary_cross_entropy = 1.019074
Epoch 127
Validation binary_cross_entropy = 0.991051
Epoch 128
Validation binary_cross_entropy = 0.999300
Epoch 129
Loss = 2.9673e-02, PNorm = 91.2952, GNorm = 1.2887, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 1.016703
Epoch 130
Validation binary_cross_entropy = 1.097478
Epoch 131
Validation binary_cross_entropy = 1.123731
Epoch 132
Validation binary_cross_entropy = 1.176865
Epoch 133
Validation binary_cross_entropy = 1.216215
Epoch 134
Loss = 2.2931e-03, PNorm = 91.3919, GNorm = 0.0337, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 1.336551
Epoch 135
Validation binary_cross_entropy = 1.443239
Epoch 136
Validation binary_cross_entropy = 1.537221
Epoch 137
Validation binary_cross_entropy = 1.454637
Epoch 138
Validation binary_cross_entropy = 1.344706
Epoch 139
Loss = 2.9199e-02, PNorm = 91.5148, GNorm = 1.4103, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 1.347035
Epoch 140
Validation binary_cross_entropy = 1.296911
Epoch 141
Validation binary_cross_entropy = 1.235683
Epoch 142
Validation binary_cross_entropy = 1.211084
Epoch 143
Validation binary_cross_entropy = 1.228173
Epoch 144
Loss = 5.6398e-03, PNorm = 91.6514, GNorm = 0.3323, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.235299
Epoch 145
Validation binary_cross_entropy = 1.219610
Epoch 146
Validation binary_cross_entropy = 1.204254
Epoch 147
Validation binary_cross_entropy = 1.209091
Epoch 148
Validation binary_cross_entropy = 1.279625
Epoch 149
Loss = 5.8947e-03, PNorm = 91.7411, GNorm = 0.2620, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.343739
Epoch 150
Validation binary_cross_entropy = 1.361551
Epoch 151
Validation binary_cross_entropy = 1.403709
Epoch 152
Validation binary_cross_entropy = 1.512502
Epoch 153
Validation binary_cross_entropy = 1.444098
Epoch 154
Loss = 1.8033e-02, PNorm = 91.8042, GNorm = 1.5142, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 1.435648
Epoch 155
Validation binary_cross_entropy = 1.425885
Epoch 156
Validation binary_cross_entropy = 1.413105
Epoch 157
Validation binary_cross_entropy = 1.423285
Epoch 158
Validation binary_cross_entropy = 1.490476
Epoch 159
Loss = 9.4308e-03, PNorm = 91.8739, GNorm = 0.6308, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.472574
Epoch 160
Validation binary_cross_entropy = 1.494928
Epoch 161
Validation binary_cross_entropy = 1.510949
Epoch 162
Validation binary_cross_entropy = 1.553056
Epoch 163
Validation binary_cross_entropy = 1.547364
Epoch 164
Loss = 3.6086e-02, PNorm = 91.9551, GNorm = 1.3878, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.546798
Epoch 165
Validation binary_cross_entropy = 1.488395
Epoch 166
Validation binary_cross_entropy = 1.497344
Epoch 167
Validation binary_cross_entropy = 1.506657
Epoch 168
Validation binary_cross_entropy = 1.815842
Epoch 169
Loss = 8.8562e-02, PNorm = 92.0357, GNorm = 0.0377, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.378586
Epoch 170
Validation binary_cross_entropy = 1.186990
Epoch 171
Validation binary_cross_entropy = 1.003239
Epoch 172
Validation binary_cross_entropy = 0.910956
Epoch 173
Validation binary_cross_entropy = 0.815459
Epoch 174
Loss = 6.2460e-02, PNorm = 92.2077, GNorm = 1.0611, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 0.772248
Epoch 175
Validation binary_cross_entropy = 0.660818
Epoch 176
Validation binary_cross_entropy = 0.628198
Epoch 177
Validation binary_cross_entropy = 0.672699
Epoch 178
Validation binary_cross_entropy = 0.663484
Epoch 179
Loss = 3.8512e-02, PNorm = 92.4241, GNorm = 0.2358, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 0.696972
Epoch 180
Validation binary_cross_entropy = 0.734299
Epoch 181
Validation binary_cross_entropy = 0.790780
Epoch 182
Validation binary_cross_entropy = 0.843736
Epoch 183
Validation binary_cross_entropy = 0.908554
Epoch 184
Loss = 4.2537e-02, PNorm = 92.6427, GNorm = 0.4229, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 0.968230
Epoch 185
Validation binary_cross_entropy = 1.027330
Epoch 186
Validation binary_cross_entropy = 1.082945
Epoch 187
Validation binary_cross_entropy = 1.126564
Epoch 188
Validation binary_cross_entropy = 1.179410
Epoch 189
Loss = 2.5223e-02, PNorm = 92.7946, GNorm = 0.6206, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.207916
Epoch 190
Validation binary_cross_entropy = 1.280986
Epoch 191
Validation binary_cross_entropy = 1.264115
Epoch 192
Validation binary_cross_entropy = 1.259869
Epoch 193
Validation binary_cross_entropy = 1.277932
Epoch 194
Loss = 8.3371e-03, PNorm = 92.9205, GNorm = 0.7858, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.312780
Epoch 195
Validation binary_cross_entropy = 1.382598
Epoch 196
Validation binary_cross_entropy = 1.431080
Epoch 197
Validation binary_cross_entropy = 1.405775
Epoch 198
Validation binary_cross_entropy = 1.359193
Epoch 199
Loss = 9.7831e-04, PNorm = 93.0070, GNorm = 0.0664, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.346297
Epoch 200
Validation binary_cross_entropy = 1.366382
Epoch 201
Validation binary_cross_entropy = 1.421153
Epoch 202
Validation binary_cross_entropy = 1.527538
Epoch 203
Validation binary_cross_entropy = 1.628461
Epoch 204
Loss = 2.2185e-02, PNorm = 93.0830, GNorm = 0.9394, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.595544
Epoch 205
Validation binary_cross_entropy = 1.532364
Epoch 206
Validation binary_cross_entropy = 1.519168
Epoch 207
Validation binary_cross_entropy = 1.538928
Epoch 208
Validation binary_cross_entropy = 1.563234
Epoch 209
Loss = 3.5316e-04, PNorm = 93.1368, GNorm = 0.0285, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.584325
Epoch 210
Validation binary_cross_entropy = 1.560305
Epoch 211
Validation binary_cross_entropy = 1.562965
Epoch 212
Validation binary_cross_entropy = 1.710925
Epoch 213
Validation binary_cross_entropy = 1.654407
Epoch 214
Loss = 4.7588e-03, PNorm = 93.1745, GNorm = 0.3315, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.492909
Epoch 215
Validation binary_cross_entropy = 1.364253
Epoch 216
Validation binary_cross_entropy = 1.297065
Epoch 217
Validation binary_cross_entropy = 1.258299
Epoch 218
Validation binary_cross_entropy = 1.228463
Epoch 219
Loss = 3.3293e-03, PNorm = 93.2114, GNorm = 0.0121, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.214431
Epoch 220
Validation binary_cross_entropy = 1.226247
Epoch 221
Validation binary_cross_entropy = 1.282156
Epoch 222
Validation binary_cross_entropy = 1.322176
Epoch 223
Validation binary_cross_entropy = 1.324849
Epoch 224
Loss = 3.3594e-03, PNorm = 93.2579, GNorm = 0.1143, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.310070
Epoch 225
Validation binary_cross_entropy = 1.295157
Epoch 226
Validation binary_cross_entropy = 1.286461
Epoch 227
Validation binary_cross_entropy = 1.288040
Epoch 228
Validation binary_cross_entropy = 1.298355
Epoch 229
Loss = 3.2333e-03, PNorm = 93.2956, GNorm = 0.2048, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.311333
Epoch 230
Validation binary_cross_entropy = 1.336538
Epoch 231
Validation binary_cross_entropy = 1.445569
Epoch 232
Validation binary_cross_entropy = 1.549495
Epoch 233
Validation binary_cross_entropy = 1.478038
Epoch 234
Loss = 1.5175e-03, PNorm = 93.3372, GNorm = 0.0197, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.401468
Epoch 235
Validation binary_cross_entropy = 1.355990
Epoch 236
Validation binary_cross_entropy = 1.330384
Epoch 237
Validation binary_cross_entropy = 1.387155
Epoch 238
Validation binary_cross_entropy = 1.440676
Epoch 239
Loss = 2.2727e-03, PNorm = 93.3908, GNorm = 0.0187, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.502336
Epoch 240
Validation binary_cross_entropy = 1.569715
Epoch 241
Validation binary_cross_entropy = 1.624342
Epoch 242
Validation binary_cross_entropy = 1.651577
Epoch 243
Validation binary_cross_entropy = 1.641945
Epoch 244
Loss = 6.8174e-04, PNorm = 93.4273, GNorm = 0.0157, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.632850
Epoch 245
Validation binary_cross_entropy = 1.626477
Epoch 246
Validation binary_cross_entropy = 1.622033
Epoch 247
Validation binary_cross_entropy = 1.618983
Epoch 248
Validation binary_cross_entropy = 1.620397
Epoch 249
Loss = 6.6245e-04, PNorm = 93.4487, GNorm = 0.0752, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.624024
Epoch 250
Validation binary_cross_entropy = 1.629071
Epoch 251
Validation binary_cross_entropy = 1.634365
Epoch 252
Validation binary_cross_entropy = 1.640249
Epoch 253
Validation binary_cross_entropy = 1.648716
Epoch 254
Loss = 1.4518e-04, PNorm = 93.4628, GNorm = 0.0031, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.658715
Epoch 255
Validation binary_cross_entropy = 1.667130
Epoch 256
Validation binary_cross_entropy = 1.674077
Epoch 257
Validation binary_cross_entropy = 1.679589
Epoch 258
Validation binary_cross_entropy = 1.683860
Epoch 259
Loss = 2.0314e-04, PNorm = 93.4720, GNorm = 0.0062, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.687803
Epoch 260
Validation binary_cross_entropy = 1.690891
Epoch 261
Validation binary_cross_entropy = 1.693186
Epoch 262
Validation binary_cross_entropy = 1.694883
Epoch 263
Validation binary_cross_entropy = 1.696245
Epoch 264
Loss = 1.4292e-04, PNorm = 93.4772, GNorm = 0.0027, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.697472
Epoch 265
Validation binary_cross_entropy = 1.699000
Epoch 266
Validation binary_cross_entropy = 1.701099
Epoch 267
Validation binary_cross_entropy = 1.703282
Epoch 268
Validation binary_cross_entropy = 1.705153
Epoch 269
Loss = 1.8177e-04, PNorm = 93.4813, GNorm = 0.0025, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.707254
Epoch 270
Validation binary_cross_entropy = 1.709727
Epoch 271
Validation binary_cross_entropy = 1.711672
Epoch 272
Validation binary_cross_entropy = 1.713488
Epoch 273
Validation binary_cross_entropy = 1.715009
Epoch 274
Loss = 1.3461e-04, PNorm = 93.4847, GNorm = 0.0020, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.716772
Epoch 275
Validation binary_cross_entropy = 1.718697
Epoch 276
Validation binary_cross_entropy = 1.720907
Epoch 277
Validation binary_cross_entropy = 1.723466
Epoch 278
Validation binary_cross_entropy = 1.725576
Epoch 279
Loss = 1.0492e-04, PNorm = 93.4881, GNorm = 0.0024, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.727909
Epoch 280
Validation binary_cross_entropy = 1.730454
Epoch 281
Validation binary_cross_entropy = 1.733060
Epoch 282
Validation binary_cross_entropy = 1.735237
Epoch 283
Validation binary_cross_entropy = 1.737036
Epoch 284
Loss = 1.2544e-04, PNorm = 93.4915, GNorm = 0.0027, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.739299
Epoch 285
Validation binary_cross_entropy = 1.742139
Epoch 286
Validation binary_cross_entropy = 1.745195
Epoch 287
Validation binary_cross_entropy = 1.747722
Epoch 288
Validation binary_cross_entropy = 1.749780
Epoch 289
Loss = 7.6081e-05, PNorm = 93.4952, GNorm = 0.0017, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.751838
Epoch 290
Validation binary_cross_entropy = 1.753638
Epoch 291
Validation binary_cross_entropy = 1.755202
Epoch 292
Validation binary_cross_entropy = 1.758588
Epoch 293
Validation binary_cross_entropy = 1.761710
Epoch 294
Loss = 1.0199e-04, PNorm = 93.4990, GNorm = 0.0046, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.764652
Epoch 295
Validation binary_cross_entropy = 1.767444
Epoch 296
Validation binary_cross_entropy = 1.770454
Epoch 297
Validation binary_cross_entropy = 1.774397
Epoch 298
Validation binary_cross_entropy = 1.780081
Epoch 299
Loss = 5.2067e-05, PNorm = 93.5036, GNorm = 0.0014, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.787435
Model 0 best validation binary_cross_entropy = 0.225167 on epoch 24
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.197010
Ensemble test binary_cross_entropy = 0.197010
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1700, bias=False)
        (W_h): Linear(in_features=1700, out_features=1700, bias=False)
        (W_o): Linear(in_features=1833, out_features=1700, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=1700, out_features=1700, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=1700, out_features=1700, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.0, inplace=False)
    (7): Linear(in_features=1700, out_features=1, bias=True)
  )
)
Number of parameters = 12,042,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.631783
Epoch 1
Validation binary_cross_entropy = 0.676069
Epoch 2
Validation binary_cross_entropy = 0.658767
Epoch 3
Validation binary_cross_entropy = 0.743657
Epoch 4
Loss = 6.4000e-01, PNorm = 84.5270, GNorm = 0.3318, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.449394
Epoch 5
Validation binary_cross_entropy = 0.389520
Epoch 6
Validation binary_cross_entropy = 0.635232
Epoch 7
Validation binary_cross_entropy = 0.577275
Epoch 8
Validation binary_cross_entropy = 0.884148
Epoch 9
Loss = 5.7658e-01, PNorm = 84.8171, GNorm = 1.4342, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.417664
Epoch 10
Validation binary_cross_entropy = 0.547969
Epoch 11
Validation binary_cross_entropy = 0.419412
Epoch 12
Validation binary_cross_entropy = 0.458979
Epoch 13
Validation binary_cross_entropy = 0.403077
Epoch 14
Loss = 5.2211e-01, PNorm = 85.1965, GNorm = 0.3027, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.365795
Epoch 15
Validation binary_cross_entropy = 0.422211
Epoch 16
Validation binary_cross_entropy = 0.405870
Epoch 17
Validation binary_cross_entropy = 0.469480
Epoch 18
Validation binary_cross_entropy = 0.346597
Epoch 19
Loss = 4.1625e-01, PNorm = 85.6110, GNorm = 1.3619, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.261200
Epoch 20
Validation binary_cross_entropy = 0.710501
Epoch 21
Validation binary_cross_entropy = 0.236773
Epoch 22
Validation binary_cross_entropy = 0.317155
Epoch 23
Validation binary_cross_entropy = 0.510288
Epoch 24
Loss = 3.9977e-01, PNorm = 85.9994, GNorm = 0.6319, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.491096
Epoch 25
Validation binary_cross_entropy = 0.325711
Epoch 26
Validation binary_cross_entropy = 0.338825
Epoch 27
Validation binary_cross_entropy = 0.344543
Epoch 28
Validation binary_cross_entropy = 0.329920
Epoch 29
Loss = 3.7299e-01, PNorm = 86.4025, GNorm = 0.5844, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.339296
Epoch 30
Validation binary_cross_entropy = 0.319745
Epoch 31
Validation binary_cross_entropy = 0.458761
Epoch 32
Validation binary_cross_entropy = 0.323823
Epoch 33
Validation binary_cross_entropy = 0.557752
Epoch 34
Loss = 4.7349e-01, PNorm = 86.7721, GNorm = 1.8008, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.301953
Epoch 35
Validation binary_cross_entropy = 0.379623
Epoch 36
Validation binary_cross_entropy = 0.613172
Epoch 37
Validation binary_cross_entropy = 0.402957
Epoch 38
Validation binary_cross_entropy = 0.309070
Epoch 39
Loss = 3.6725e-01, PNorm = 87.1247, GNorm = 0.8010, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.344646
Epoch 40
Validation binary_cross_entropy = 0.419174
Epoch 41
Validation binary_cross_entropy = 0.374229
Epoch 42
Validation binary_cross_entropy = 0.316643
Epoch 43
Validation binary_cross_entropy = 0.430354
Epoch 44
Loss = 2.0259e-01, PNorm = 87.5409, GNorm = 0.7059, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.483046
Epoch 45
Validation binary_cross_entropy = 0.432997
Epoch 46
Validation binary_cross_entropy = 0.544566
Epoch 47
Validation binary_cross_entropy = 0.454700
Epoch 48
Validation binary_cross_entropy = 0.472998
Epoch 49
Loss = 1.4945e-01, PNorm = 87.9697, GNorm = 1.0022, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.416516
Epoch 50
Validation binary_cross_entropy = 0.419122
Epoch 51
Validation binary_cross_entropy = 0.475439
Epoch 52
Validation binary_cross_entropy = 0.450709
Epoch 53
Validation binary_cross_entropy = 0.458409
Epoch 54
Loss = 1.5343e-01, PNorm = 88.3702, GNorm = 0.7466, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.504009
Epoch 55
Validation binary_cross_entropy = 0.475295
Epoch 56
Validation binary_cross_entropy = 0.521749
Epoch 57
Validation binary_cross_entropy = 0.559299
Epoch 58
Validation binary_cross_entropy = 0.569559
Epoch 59
Loss = 1.2099e-01, PNorm = 88.7256, GNorm = 0.4024, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.673546
Epoch 60
Validation binary_cross_entropy = 0.633027
Epoch 61
Validation binary_cross_entropy = 0.582999
Epoch 62
Validation binary_cross_entropy = 0.615752
Epoch 63
Validation binary_cross_entropy = 0.788089
Epoch 64
Loss = 2.0250e-01, PNorm = 89.0564, GNorm = 1.2775, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.711756
Epoch 65
Validation binary_cross_entropy = 0.623253
Epoch 66
Validation binary_cross_entropy = 0.703009
Epoch 67
Validation binary_cross_entropy = 0.646053
Epoch 68
Validation binary_cross_entropy = 0.627560
Epoch 69
Loss = 1.2253e-01, PNorm = 89.3456, GNorm = 0.6573, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.556689
Epoch 70
Validation binary_cross_entropy = 0.559173
Epoch 71
Validation binary_cross_entropy = 0.607402
Epoch 72
Validation binary_cross_entropy = 0.655261
Epoch 73
Validation binary_cross_entropy = 0.720289
Epoch 74
Loss = 5.0666e-02, PNorm = 89.6798, GNorm = 0.4849, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.807237
Epoch 75
Validation binary_cross_entropy = 1.013448
Epoch 76
Validation binary_cross_entropy = 0.994031
Epoch 77
Validation binary_cross_entropy = 1.014668
Epoch 78
Validation binary_cross_entropy = 1.048726
Epoch 79
Loss = 1.7096e-02, PNorm = 89.9592, GNorm = 0.5320, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.964680
Epoch 80
Validation binary_cross_entropy = 0.980872
Epoch 81
Validation binary_cross_entropy = 0.822316
Epoch 82
Validation binary_cross_entropy = 0.724186
Epoch 83
Validation binary_cross_entropy = 0.674673
Epoch 84
Loss = 8.9626e-02, PNorm = 90.1785, GNorm = 0.7038, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.580881
Epoch 85
Validation binary_cross_entropy = 0.558237
Epoch 86
Validation binary_cross_entropy = 0.540765
Epoch 87
Validation binary_cross_entropy = 0.581185
Epoch 88
Validation binary_cross_entropy = 0.598103
Epoch 89
Loss = 5.5360e-02, PNorm = 90.4307, GNorm = 0.3902, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.622011
Epoch 90
Validation binary_cross_entropy = 0.705934
Epoch 91
Validation binary_cross_entropy = 0.755535
Epoch 92
Validation binary_cross_entropy = 0.910694
Epoch 93
Validation binary_cross_entropy = 0.870221
Epoch 94
Loss = 4.7514e-02, PNorm = 90.6819, GNorm = 0.3034, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.833093
Epoch 95
Validation binary_cross_entropy = 0.825300
Epoch 96
Validation binary_cross_entropy = 0.844611
Epoch 97
Validation binary_cross_entropy = 0.874564
Epoch 98
Validation binary_cross_entropy = 0.955924
Epoch 99
Loss = 9.7704e-02, PNorm = 90.8801, GNorm = 1.7321, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.999754
Epoch 100
Validation binary_cross_entropy = 1.014398
Epoch 101
Validation binary_cross_entropy = 1.049435
Epoch 102
Validation binary_cross_entropy = 1.063013
Epoch 103
Validation binary_cross_entropy = 1.030936
Epoch 104
Loss = 5.5003e-03, PNorm = 91.0204, GNorm = 0.3006, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.995300
Epoch 105
Validation binary_cross_entropy = 0.987322
Epoch 106
Validation binary_cross_entropy = 0.993437
Epoch 107
Validation binary_cross_entropy = 0.982225
Epoch 108
Validation binary_cross_entropy = 1.015611
Epoch 109
Loss = 1.9464e-02, PNorm = 91.1335, GNorm = 0.3187, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 1.031705
Epoch 110
Validation binary_cross_entropy = 1.047267
Epoch 111
Validation binary_cross_entropy = 1.012605
Epoch 112
Validation binary_cross_entropy = 1.023056
Epoch 113
Validation binary_cross_entropy = 1.064025
Epoch 114
Loss = 5.6521e-03, PNorm = 91.2444, GNorm = 0.3389, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 1.141104
Epoch 115
Validation binary_cross_entropy = 1.262331
Epoch 116
Validation binary_cross_entropy = 1.264004
Epoch 117
Validation binary_cross_entropy = 1.303644
Epoch 118
Validation binary_cross_entropy = 1.348240
Epoch 119
Loss = 1.6608e-03, PNorm = 91.3497, GNorm = 0.0735, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 1.389946
Epoch 120
Validation binary_cross_entropy = 1.428459
Epoch 121
Validation binary_cross_entropy = 1.465641
Epoch 122
Validation binary_cross_entropy = 1.584502
Epoch 123
Validation binary_cross_entropy = 1.699054
Epoch 124
Loss = 4.8890e-02, PNorm = 91.4231, GNorm = 4.0598, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 1.399495
Epoch 125
Validation binary_cross_entropy = 1.550645
Epoch 126
Validation binary_cross_entropy = 1.259933
Epoch 127
Validation binary_cross_entropy = 1.225678
Epoch 128
Validation binary_cross_entropy = 1.166328
Epoch 129
Loss = 8.9748e-02, PNorm = 91.6286, GNorm = 1.8904, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 1.189062
Epoch 130
Validation binary_cross_entropy = 1.289810
Epoch 131
Validation binary_cross_entropy = 1.887463
Epoch 132
Validation binary_cross_entropy = 1.013631
Epoch 133
Validation binary_cross_entropy = 1.328971
Epoch 134
Loss = 5.3894e-01, PNorm = 92.0183, GNorm = 2.3983, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.656812
Epoch 135
Validation binary_cross_entropy = 0.715036
Epoch 136
Validation binary_cross_entropy = 0.572600
Epoch 137
Validation binary_cross_entropy = 0.411531
Epoch 138
Validation binary_cross_entropy = 0.345713
Epoch 139
Loss = 2.9343e-01, PNorm = 92.6171, GNorm = 0.5923, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.360996
Epoch 140
Validation binary_cross_entropy = 0.407821
Epoch 141
Validation binary_cross_entropy = 0.502606
Epoch 142
Validation binary_cross_entropy = 0.574997
Epoch 143
Validation binary_cross_entropy = 0.673117
Epoch 144
Loss = 1.0572e-01, PNorm = 93.1758, GNorm = 0.8038, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.709101
Epoch 145
Validation binary_cross_entropy = 0.769203
Epoch 146
Validation binary_cross_entropy = 0.887657
Epoch 147
Validation binary_cross_entropy = 1.012825
Epoch 148
Validation binary_cross_entropy = 1.062607
Epoch 149
Loss = 2.5537e-02, PNorm = 93.6364, GNorm = 0.2005, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.127950
Epoch 150
Validation binary_cross_entropy = 1.209918
Epoch 151
Validation binary_cross_entropy = 1.288823
Epoch 152
Validation binary_cross_entropy = 1.293668
Epoch 153
Validation binary_cross_entropy = 1.336310
Epoch 154
Loss = 5.3931e-02, PNorm = 93.8898, GNorm = 1.7754, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 1.260090
Epoch 155
Validation binary_cross_entropy = 1.204472
Epoch 156
Validation binary_cross_entropy = 1.292242
Epoch 157
Validation binary_cross_entropy = 1.360480
Epoch 158
Validation binary_cross_entropy = 1.321801
Epoch 159
Loss = 3.1418e-02, PNorm = 94.0648, GNorm = 1.0105, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.264397
Epoch 160
Validation binary_cross_entropy = 1.239606
Epoch 161
Validation binary_cross_entropy = 1.237643
Epoch 162
Validation binary_cross_entropy = 1.223265
Epoch 163
Validation binary_cross_entropy = 1.212568
Epoch 164
Loss = 3.4405e-02, PNorm = 94.1949, GNorm = 0.8401, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.222916
Epoch 165
Validation binary_cross_entropy = 1.255726
Epoch 166
Validation binary_cross_entropy = 1.294479
Epoch 167
Validation binary_cross_entropy = 1.277980
Epoch 168
Validation binary_cross_entropy = 1.257566
Epoch 169
Loss = 5.4390e-03, PNorm = 94.2834, GNorm = 0.0412, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.271974
Epoch 170
Validation binary_cross_entropy = 1.300980
Epoch 171
Validation binary_cross_entropy = 1.361714
Epoch 172
Validation binary_cross_entropy = 1.426332
Epoch 173
Validation binary_cross_entropy = 1.499264
Epoch 174
Loss = 1.2875e-02, PNorm = 94.3804, GNorm = 0.6106, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.601142
Epoch 175
Validation binary_cross_entropy = 1.638561
Epoch 176
Validation binary_cross_entropy = 1.598764
Epoch 177
Validation binary_cross_entropy = 1.569783
Epoch 178
Validation binary_cross_entropy = 1.559052
Epoch 179
Loss = 2.1830e-03, PNorm = 94.4522, GNorm = 0.2444, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.572806
Epoch 180
Validation binary_cross_entropy = 1.621299
Epoch 181
Validation binary_cross_entropy = 1.684641
Epoch 182
Validation binary_cross_entropy = 1.703131
Epoch 183
Validation binary_cross_entropy = 1.694657
Epoch 184
Loss = 1.5670e-03, PNorm = 94.5111, GNorm = 0.0282, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.679446
Epoch 185
Validation binary_cross_entropy = 1.665754
Epoch 186
Validation binary_cross_entropy = 1.653664
Epoch 187
Validation binary_cross_entropy = 1.643519
Epoch 188
Validation binary_cross_entropy = 1.632653
Epoch 189
Loss = 6.1316e-04, PNorm = 94.5504, GNorm = 0.0099, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.620386
Epoch 190
Validation binary_cross_entropy = 1.625765
Epoch 191
Validation binary_cross_entropy = 1.646288
Epoch 192
Validation binary_cross_entropy = 1.676473
Epoch 193
Validation binary_cross_entropy = 1.715518
Epoch 194
Loss = 3.2389e-03, PNorm = 94.5750, GNorm = 0.3177, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.767555
Epoch 195
Validation binary_cross_entropy = 1.823441
Epoch 196
Validation binary_cross_entropy = 1.895959
Epoch 197
Validation binary_cross_entropy = 1.919995
Epoch 198
Validation binary_cross_entropy = 1.876729
Epoch 199
Loss = 2.7888e-03, PNorm = 94.6262, GNorm = 0.0199, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.840530
Epoch 200
Validation binary_cross_entropy = 1.854485
Epoch 201
Validation binary_cross_entropy = 1.897732
Epoch 202
Validation binary_cross_entropy = 1.929275
Epoch 203
Validation binary_cross_entropy = 1.939687
Epoch 204
Loss = 6.8260e-03, PNorm = 94.6947, GNorm = 0.0670, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 2.007064
Epoch 205
Validation binary_cross_entropy = 2.056159
Epoch 206
Validation binary_cross_entropy = 2.076960
Epoch 207
Validation binary_cross_entropy = 2.077188
Epoch 208
Validation binary_cross_entropy = 2.148840
Epoch 209
Loss = 1.7876e-03, PNorm = 94.7567, GNorm = 0.1288, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 2.198178
Epoch 210
Validation binary_cross_entropy = 2.211438
Epoch 211
Validation binary_cross_entropy = 2.195218
Epoch 212
Validation binary_cross_entropy = 2.154576
Epoch 213
Validation binary_cross_entropy = 2.117762
Epoch 214
Loss = 5.3456e-04, PNorm = 94.8055, GNorm = 0.0092, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 2.110081
Epoch 215
Validation binary_cross_entropy = 2.232212
Epoch 216
Validation binary_cross_entropy = 2.608691
Epoch 217
Validation binary_cross_entropy = 2.081024
Epoch 218
Validation binary_cross_entropy = 1.920689
Epoch 219
Loss = 3.1701e-01, PNorm = 94.8603, GNorm = 6.1627, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.739135
Epoch 220
Validation binary_cross_entropy = 1.438473
Epoch 221
Validation binary_cross_entropy = 1.520429
Epoch 222
Validation binary_cross_entropy = 1.274948
Epoch 223
Validation binary_cross_entropy = 1.105358
Epoch 224
Loss = 5.0596e-02, PNorm = 95.1058, GNorm = 0.6213, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.028772
Epoch 225
Validation binary_cross_entropy = 0.999100
Epoch 226
Validation binary_cross_entropy = 0.990181
Epoch 227
Validation binary_cross_entropy = 1.050192
Epoch 228
Validation binary_cross_entropy = 1.098241
Epoch 229
Loss = 2.0012e-02, PNorm = 95.3712, GNorm = 0.2715, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.155934
Epoch 230
Validation binary_cross_entropy = 1.230517
Epoch 231
Validation binary_cross_entropy = 1.316691
Epoch 232
Validation binary_cross_entropy = 1.404503
Epoch 233
Validation binary_cross_entropy = 1.542650
Epoch 234
Loss = 2.5622e-02, PNorm = 95.5763, GNorm = 0.7363, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.677441
Epoch 235
Validation binary_cross_entropy = 1.704821
Epoch 236
Validation binary_cross_entropy = 1.692999
Epoch 237
Validation binary_cross_entropy = 1.681406
Epoch 238
Validation binary_cross_entropy = 1.672928
Epoch 239
Loss = 1.0042e-02, PNorm = 95.7311, GNorm = 0.7841, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.694111
Epoch 240
Validation binary_cross_entropy = 1.780046
Epoch 241
Validation binary_cross_entropy = 1.839319
Epoch 242
Validation binary_cross_entropy = 1.926808
Epoch 243
Validation binary_cross_entropy = 1.897288
Epoch 244
Loss = 5.2697e-03, PNorm = 95.8589, GNorm = 0.2887, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.857323
Epoch 245
Validation binary_cross_entropy = 1.822374
Epoch 246
Validation binary_cross_entropy = 1.801025
Epoch 247
Validation binary_cross_entropy = 1.790304
Epoch 248
Validation binary_cross_entropy = 1.827657
Epoch 249
Loss = 2.3370e-03, PNorm = 95.9490, GNorm = 0.2124, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.863314
Epoch 250
Validation binary_cross_entropy = 1.943722
Epoch 251
Validation binary_cross_entropy = 2.031973
Epoch 252
Validation binary_cross_entropy = 2.002774
Epoch 253
Validation binary_cross_entropy = 1.969943
Epoch 254
Loss = 1.1293e-03, PNorm = 96.0172, GNorm = 0.0249, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.937367
Epoch 255
Validation binary_cross_entropy = 1.960832
Epoch 256
Validation binary_cross_entropy = 1.983346
Epoch 257
Validation binary_cross_entropy = 2.004286
Epoch 258
Validation binary_cross_entropy = 2.035648
Epoch 259
Loss = 7.0048e-04, PNorm = 96.0727, GNorm = 0.0185, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 2.058105
Epoch 260
Validation binary_cross_entropy = 2.074057
Epoch 261
Validation binary_cross_entropy = 2.091892
Epoch 262
Validation binary_cross_entropy = 2.104531
Epoch 263
Validation binary_cross_entropy = 2.132811
Epoch 264
Loss = 3.9876e-04, PNorm = 96.1106, GNorm = 0.0116, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 2.155611
Epoch 265
Validation binary_cross_entropy = 2.174240
Epoch 266
Validation binary_cross_entropy = 2.258204
Epoch 267
Validation binary_cross_entropy = 2.326425
Epoch 268
Validation binary_cross_entropy = 2.320725
Epoch 269
Loss = 3.1260e-03, PNorm = 96.1495, GNorm = 0.1526, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 2.276307
Epoch 270
Validation binary_cross_entropy = 2.215036
Epoch 271
Validation binary_cross_entropy = 2.167910
Epoch 272
Validation binary_cross_entropy = 2.133544
Epoch 273
Validation binary_cross_entropy = 2.138430
Epoch 274
Loss = 6.1242e-02, PNorm = 96.1830, GNorm = 1.1944, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 2.170309
Epoch 275
Validation binary_cross_entropy = 2.273017
Epoch 276
Validation binary_cross_entropy = 2.288883
Epoch 277
Validation binary_cross_entropy = 2.184219
Epoch 278
Validation binary_cross_entropy = 2.100030
Epoch 279
Loss = 3.6004e-04, PNorm = 96.2031, GNorm = 0.0059, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 2.044079
Epoch 280
Validation binary_cross_entropy = 2.012961
Epoch 281
Validation binary_cross_entropy = 2.001945
Epoch 282
Validation binary_cross_entropy = 2.023111
Epoch 283
Validation binary_cross_entropy = 2.049331
Epoch 284
Loss = 2.6486e-04, PNorm = 96.2335, GNorm = 0.0131, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 2.077898
Epoch 285
Validation binary_cross_entropy = 2.219001
Epoch 286
Validation binary_cross_entropy = 2.359042
Epoch 287
Validation binary_cross_entropy = 2.282156
Epoch 288
Validation binary_cross_entropy = 2.123439
Epoch 289
Loss = 1.5270e-04, PNorm = 96.2543, GNorm = 0.0028, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 2.028966
Epoch 290
Validation binary_cross_entropy = 1.963763
Epoch 291
Validation binary_cross_entropy = 1.981497
Epoch 292
Validation binary_cross_entropy = 2.019451
Epoch 293
Validation binary_cross_entropy = 2.068938
Epoch 294
Loss = 1.3646e-03, PNorm = 96.2765, GNorm = 0.1127, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 2.100586
Epoch 295
Validation binary_cross_entropy = 2.086482
Epoch 296
Validation binary_cross_entropy = 2.064875
Epoch 297
Validation binary_cross_entropy = 2.051142
Epoch 298
Validation binary_cross_entropy = 2.043918
Epoch 299
Loss = 9.4416e-04, PNorm = 96.2920, GNorm = 0.0073, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 2.054797
Model 0 best validation binary_cross_entropy = 0.236773 on epoch 21
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.210249
Ensemble test binary_cross_entropy = 0.210249
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1700, bias=False)
        (W_h): Linear(in_features=1700, out_features=1700, bias=False)
        (W_o): Linear(in_features=1833, out_features=1700, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=1700, out_features=1700, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=1700, out_features=1700, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.0, inplace=False)
    (7): Linear(in_features=1700, out_features=1, bias=True)
  )
)
Number of parameters = 12,042,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.535857
Epoch 1
Validation binary_cross_entropy = 0.730517
Epoch 2
Validation binary_cross_entropy = 0.612910
Epoch 3
Validation binary_cross_entropy = 0.786216
Epoch 4
Loss = 6.5411e-01, PNorm = 84.5260, GNorm = 0.4091, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.479843
Epoch 5
Validation binary_cross_entropy = 0.483120
Epoch 6
Validation binary_cross_entropy = 0.932908
Epoch 7
Validation binary_cross_entropy = 0.681304
Epoch 8
Validation binary_cross_entropy = 0.995330
Epoch 9
Loss = 6.0411e-01, PNorm = 84.8063, GNorm = 1.1215, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.427377
Epoch 10
Validation binary_cross_entropy = 0.475119
Epoch 11
Validation binary_cross_entropy = 0.610694
Epoch 12
Validation binary_cross_entropy = 0.650930
Epoch 13
Validation binary_cross_entropy = 0.441047
Epoch 14
Loss = 4.7912e-01, PNorm = 85.1885, GNorm = 0.7599, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.489891
Epoch 15
Validation binary_cross_entropy = 0.266847
Epoch 16
Validation binary_cross_entropy = 0.633168
Epoch 17
Validation binary_cross_entropy = 0.298215
Epoch 18
Validation binary_cross_entropy = 0.458436
Epoch 19
Loss = 4.2617e-01, PNorm = 85.5756, GNorm = 0.6799, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.577191
Epoch 20
Validation binary_cross_entropy = 0.311387
Epoch 21
Validation binary_cross_entropy = 0.351731
Epoch 22
Validation binary_cross_entropy = 0.387555
Epoch 23
Validation binary_cross_entropy = 0.430720
Epoch 24
Loss = 3.3501e-01, PNorm = 85.9820, GNorm = 1.3059, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.377917
Epoch 25
Validation binary_cross_entropy = 0.457618
Epoch 26
Validation binary_cross_entropy = 0.315484
Epoch 27
Validation binary_cross_entropy = 0.453694
Epoch 28
Validation binary_cross_entropy = 0.382481
Epoch 29
Loss = 3.5172e-01, PNorm = 86.3261, GNorm = 0.6245, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.407530
Epoch 30
Validation binary_cross_entropy = 0.371913
Epoch 31
Validation binary_cross_entropy = 0.346465
Epoch 32
Validation binary_cross_entropy = 0.358999
Epoch 33
Validation binary_cross_entropy = 0.358148
Epoch 34
Loss = 3.6480e-01, PNorm = 86.7323, GNorm = 1.3114, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.539389
Epoch 35
Validation binary_cross_entropy = 0.615703
Epoch 36
Validation binary_cross_entropy = 0.631599
Epoch 37
Validation binary_cross_entropy = 0.290058
Epoch 38
Validation binary_cross_entropy = 0.260686
Epoch 39
Loss = 3.9414e-01, PNorm = 87.0918, GNorm = 0.6248, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.301147
Epoch 40
Validation binary_cross_entropy = 0.386628
Epoch 41
Validation binary_cross_entropy = 0.394394
Epoch 42
Validation binary_cross_entropy = 0.371728
Epoch 43
Validation binary_cross_entropy = 0.364315
Epoch 44
Loss = 2.3980e-01, PNorm = 87.5098, GNorm = 0.4191, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.428589
Epoch 45
Validation binary_cross_entropy = 0.386951
Epoch 46
Validation binary_cross_entropy = 0.429179
Epoch 47
Validation binary_cross_entropy = 0.500145
Epoch 48
Validation binary_cross_entropy = 0.518281
Epoch 49
Loss = 1.0513e-01, PNorm = 87.9126, GNorm = 1.0909, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.678812
Epoch 50
Validation binary_cross_entropy = 0.599434
Epoch 51
Validation binary_cross_entropy = 0.718212
Epoch 52
Validation binary_cross_entropy = 0.522545
Epoch 53
Validation binary_cross_entropy = 0.531796
Epoch 54
Loss = 2.3004e-01, PNorm = 88.2648, GNorm = 1.3122, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.453729
Epoch 55
Validation binary_cross_entropy = 0.467796
Epoch 56
Validation binary_cross_entropy = 0.489301
Epoch 57
Validation binary_cross_entropy = 0.461701
Epoch 58
Validation binary_cross_entropy = 0.489502
Epoch 59
Loss = 2.2355e-01, PNorm = 88.6460, GNorm = 1.0995, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.490306
Epoch 60
Validation binary_cross_entropy = 0.555166
Epoch 61
Validation binary_cross_entropy = 0.646162
Epoch 62
Validation binary_cross_entropy = 0.623215
Epoch 63
Validation binary_cross_entropy = 0.658433
Epoch 64
Loss = 1.0603e-01, PNorm = 88.9925, GNorm = 1.3737, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.717295
Epoch 65
Validation binary_cross_entropy = 0.723835
Epoch 66
Validation binary_cross_entropy = 0.663067
Epoch 67
Validation binary_cross_entropy = 0.553155
Epoch 68
Validation binary_cross_entropy = 0.471963
Epoch 69
Loss = 1.4020e-01, PNorm = 89.2786, GNorm = 0.6093, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.547482
Epoch 70
Validation binary_cross_entropy = 0.571305
Epoch 71
Validation binary_cross_entropy = 0.630582
Epoch 72
Validation binary_cross_entropy = 0.722192
Epoch 73
Validation binary_cross_entropy = 0.869743
Epoch 74
Loss = 9.0916e-02, PNorm = 89.6035, GNorm = 0.3183, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.709444
Epoch 75
Validation binary_cross_entropy = 0.711651
Epoch 76
Validation binary_cross_entropy = 0.758614
Epoch 77
Validation binary_cross_entropy = 0.908638
Epoch 78
Validation binary_cross_entropy = 0.749915
Epoch 79
Loss = 1.1135e-01, PNorm = 89.8934, GNorm = 0.1676, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.681183
Epoch 80
Validation binary_cross_entropy = 0.688426
Epoch 81
Validation binary_cross_entropy = 0.744312
Epoch 82
Validation binary_cross_entropy = 0.795284
Epoch 83
Validation binary_cross_entropy = 0.862030
Epoch 84
Loss = 1.7492e-01, PNorm = 90.1306, GNorm = 0.8049, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.853690
Epoch 85
Validation binary_cross_entropy = 0.836794
Epoch 86
Validation binary_cross_entropy = 0.773291
Epoch 87
Validation binary_cross_entropy = 0.756828
Epoch 88
Validation binary_cross_entropy = 0.807945
Epoch 89
Loss = 5.7400e-02, PNorm = 90.3244, GNorm = 1.1082, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.799082
Epoch 90
Validation binary_cross_entropy = 0.809227
Epoch 91
Validation binary_cross_entropy = 0.862215
Epoch 92
Validation binary_cross_entropy = 0.933760
Epoch 93
Validation binary_cross_entropy = 0.939995
Epoch 94
Loss = 1.1478e-02, PNorm = 90.5086, GNorm = 0.2254, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.999430
Epoch 95
Validation binary_cross_entropy = 1.054492
Epoch 96
Validation binary_cross_entropy = 1.097635
Epoch 97
Validation binary_cross_entropy = 1.146192
Epoch 98
Validation binary_cross_entropy = 1.090274
Epoch 99
Loss = 1.6689e-02, PNorm = 90.6879, GNorm = 0.3736, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.997738
Epoch 100
Validation binary_cross_entropy = 0.994646
Epoch 101
Validation binary_cross_entropy = 1.072915
Epoch 102
Validation binary_cross_entropy = 1.052699
Epoch 103
Validation binary_cross_entropy = 1.011158
Epoch 104
Loss = 1.8381e-02, PNorm = 90.8507, GNorm = 0.6400, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 1.033817
Epoch 105
Validation binary_cross_entropy = 1.048408
Epoch 106
Validation binary_cross_entropy = 1.109594
Epoch 107
Validation binary_cross_entropy = 1.076289
Epoch 108
Validation binary_cross_entropy = 1.103724
Epoch 109
Loss = 1.1564e-02, PNorm = 91.0554, GNorm = 0.2255, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 1.215847
Epoch 110
Validation binary_cross_entropy = 1.169250
Epoch 111
Validation binary_cross_entropy = 1.258098
Epoch 112
Validation binary_cross_entropy = 1.249956
Epoch 113
Validation binary_cross_entropy = 1.043476
Epoch 114
Loss = 1.3283e-01, PNorm = 91.2791, GNorm = 0.8212, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 0.953823
Epoch 115
Validation binary_cross_entropy = 0.959937
Epoch 116
Validation binary_cross_entropy = 0.990425
Epoch 117
Validation binary_cross_entropy = 0.974784
Epoch 118
Validation binary_cross_entropy = 0.956078
Epoch 119
Loss = 6.6600e-03, PNorm = 91.4899, GNorm = 0.1038, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 0.955762
Epoch 120
Validation binary_cross_entropy = 0.962340
Epoch 121
Validation binary_cross_entropy = 1.097599
Epoch 122
Validation binary_cross_entropy = 1.142787
Epoch 123
Validation binary_cross_entropy = 1.202080
Epoch 124
Loss = 5.9809e-03, PNorm = 91.6671, GNorm = 0.0975, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 1.215270
Epoch 125
Validation binary_cross_entropy = 1.226344
Epoch 126
Validation binary_cross_entropy = 1.233582
Epoch 127
Validation binary_cross_entropy = 1.238335
Epoch 128
Validation binary_cross_entropy = 1.238444
Epoch 129
Loss = 6.3565e-03, PNorm = 91.7929, GNorm = 0.2270, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 1.231678
Epoch 130
Validation binary_cross_entropy = 1.255303
Epoch 131
Validation binary_cross_entropy = 1.307313
Epoch 132
Validation binary_cross_entropy = 1.328201
Epoch 133
Validation binary_cross_entropy = 1.333101
Epoch 134
Loss = 5.5228e-02, PNorm = 91.8752, GNorm = 1.5316, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 1.376534
Epoch 135
Validation binary_cross_entropy = 1.292993
Epoch 136
Validation binary_cross_entropy = 1.319751
Epoch 137
Validation binary_cross_entropy = 1.131432
Epoch 138
Validation binary_cross_entropy = 1.163580
Epoch 139
Loss = 5.6134e-02, PNorm = 91.9804, GNorm = 1.1785, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.973612
Epoch 140
Validation binary_cross_entropy = 0.844628
Epoch 141
Validation binary_cross_entropy = 0.804787
Epoch 142
Validation binary_cross_entropy = 0.770662
Epoch 143
Validation binary_cross_entropy = 0.738312
Epoch 144
Loss = 1.5848e-02, PNorm = 92.1538, GNorm = 0.1122, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 0.750708
Epoch 145
Validation binary_cross_entropy = 0.788173
Epoch 146
Validation binary_cross_entropy = 0.885093
Epoch 147
Validation binary_cross_entropy = 0.958641
Epoch 148
Validation binary_cross_entropy = 1.011079
Epoch 149
Loss = 9.3118e-03, PNorm = 92.3153, GNorm = 0.1023, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.060338
Epoch 150
Validation binary_cross_entropy = 1.110614
Epoch 151
Validation binary_cross_entropy = 1.146866
Epoch 152
Validation binary_cross_entropy = 1.183984
Epoch 153
Validation binary_cross_entropy = 1.181193
Epoch 154
Loss = 4.4740e-03, PNorm = 92.4336, GNorm = 0.0734, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 1.181017
Epoch 155
Validation binary_cross_entropy = 1.213636
Epoch 156
Validation binary_cross_entropy = 1.270734
Epoch 157
Validation binary_cross_entropy = 1.328057
Epoch 158
Validation binary_cross_entropy = 1.402148
Epoch 159
Loss = 3.3735e-02, PNorm = 92.5325, GNorm = 1.9206, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.361425
Epoch 160
Validation binary_cross_entropy = 1.307470
Epoch 161
Validation binary_cross_entropy = 1.355805
Epoch 162
Validation binary_cross_entropy = 1.420700
Epoch 163
Validation binary_cross_entropy = 1.454777
Epoch 164
Loss = 1.1038e-02, PNorm = 92.6184, GNorm = 0.1773, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.415166
Epoch 165
Validation binary_cross_entropy = 1.391417
Epoch 166
Validation binary_cross_entropy = 1.413661
Epoch 167
Validation binary_cross_entropy = 1.435446
Epoch 168
Validation binary_cross_entropy = 1.455513
Epoch 169
Loss = 1.3648e-03, PNorm = 92.6944, GNorm = 0.0067, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.482525
Epoch 170
Validation binary_cross_entropy = 1.504621
Epoch 171
Validation binary_cross_entropy = 1.531299
Epoch 172
Validation binary_cross_entropy = 1.548872
Epoch 173
Validation binary_cross_entropy = 1.563010
Epoch 174
Loss = 8.5290e-04, PNorm = 92.7440, GNorm = 0.0145, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.571941
Epoch 175
Validation binary_cross_entropy = 1.578052
Epoch 176
Validation binary_cross_entropy = 1.583383
Epoch 177
Validation binary_cross_entropy = 1.584427
Epoch 178
Validation binary_cross_entropy = 1.591935
Epoch 179
Loss = 5.5173e-04, PNorm = 92.7752, GNorm = 0.0190, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.604269
Epoch 180
Validation binary_cross_entropy = 1.616261
Epoch 181
Validation binary_cross_entropy = 1.625623
Epoch 182
Validation binary_cross_entropy = 1.635127
Epoch 183
Validation binary_cross_entropy = 1.640644
Epoch 184
Loss = 3.2305e-04, PNorm = 92.7975, GNorm = 0.0103, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.644141
Epoch 185
Validation binary_cross_entropy = 1.647783
Epoch 186
Validation binary_cross_entropy = 1.652790
Epoch 187
Validation binary_cross_entropy = 1.660245
Epoch 188
Validation binary_cross_entropy = 1.675263
Epoch 189
Loss = 1.3272e-04, PNorm = 92.8158, GNorm = 0.0023, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.690637
Epoch 190
Validation binary_cross_entropy = 1.713036
Epoch 191
Validation binary_cross_entropy = 1.742206
Epoch 192
Validation binary_cross_entropy = 1.783345
Epoch 193
Validation binary_cross_entropy = 1.824343
Epoch 194
Loss = 1.8901e-04, PNorm = 92.8320, GNorm = 0.0075, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.854622
Epoch 195
Validation binary_cross_entropy = 1.864884
Epoch 196
Validation binary_cross_entropy = 1.867917
Epoch 197
Validation binary_cross_entropy = 1.868311
Epoch 198
Validation binary_cross_entropy = 1.867684
Epoch 199
Loss = 3.1921e-05, PNorm = 92.8443, GNorm = 0.0005, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.869103
Epoch 200
Validation binary_cross_entropy = 1.874794
Epoch 201
Validation binary_cross_entropy = 1.890877
Epoch 202
Validation binary_cross_entropy = 1.912410
Epoch 203
Validation binary_cross_entropy = 1.941907
Epoch 204
Loss = 1.8628e-04, PNorm = 92.8526, GNorm = 0.0364, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.981954
Epoch 205
Validation binary_cross_entropy = 2.013451
Epoch 206
Validation binary_cross_entropy = 2.015547
Epoch 207
Validation binary_cross_entropy = 1.993873
Epoch 208
Validation binary_cross_entropy = 1.964014
Epoch 209
Loss = 7.9556e-05, PNorm = 92.8591, GNorm = 0.0026, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.936797
Epoch 210
Validation binary_cross_entropy = 1.916093
Epoch 211
Validation binary_cross_entropy = 1.903760
Epoch 212
Validation binary_cross_entropy = 1.906307
Epoch 213
Validation binary_cross_entropy = 1.917391
Epoch 214
Loss = 7.0273e-05, PNorm = 92.8673, GNorm = 0.0015, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.936224
Epoch 215
Validation binary_cross_entropy = 1.953196
Epoch 216
Validation binary_cross_entropy = 1.971437
Epoch 217
Validation binary_cross_entropy = 1.992035
Epoch 218
Validation binary_cross_entropy = 2.011490
Epoch 219
Loss = 4.7275e-05, PNorm = 92.8759, GNorm = 0.0011, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 2.030102
Epoch 220
Validation binary_cross_entropy = 2.045546
Epoch 221
Validation binary_cross_entropy = 2.057598
Epoch 222
Validation binary_cross_entropy = 2.067945
Epoch 223
Validation binary_cross_entropy = 2.075816
Epoch 224
Loss = 1.8099e-05, PNorm = 92.8816, GNorm = 0.0004, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 2.082361
Epoch 225
Validation binary_cross_entropy = 2.092458
Epoch 226
Validation binary_cross_entropy = 2.100964
Epoch 227
Validation binary_cross_entropy = 2.108399
Epoch 228
Validation binary_cross_entropy = 2.114537
Epoch 229
Loss = 4.1744e-05, PNorm = 92.8847, GNorm = 0.0024, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 2.124297
Epoch 230
Validation binary_cross_entropy = 2.136186
Epoch 231
Validation binary_cross_entropy = 2.150635
Epoch 232
Validation binary_cross_entropy = 2.162092
Epoch 233
Validation binary_cross_entropy = 2.172692
Epoch 234
Loss = 1.5251e-05, PNorm = 92.8886, GNorm = 0.0008, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 2.184187
Epoch 235
Validation binary_cross_entropy = 2.193385
Epoch 236
Validation binary_cross_entropy = 2.199908
Epoch 237
Validation binary_cross_entropy = 2.207495
Epoch 238
Validation binary_cross_entropy = 2.212922
Epoch 239
Loss = 3.0794e-05, PNorm = 92.8917, GNorm = 0.0011, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 2.221124
Epoch 240
Validation binary_cross_entropy = 2.227293
Epoch 241
Validation binary_cross_entropy = 2.234834
Epoch 242
Validation binary_cross_entropy = 2.241910
Epoch 243
Validation binary_cross_entropy = 2.249598
Epoch 244
Loss = 1.4155e-05, PNorm = 92.8952, GNorm = 0.0005, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 2.259659
Epoch 245
Validation binary_cross_entropy = 2.267573
Epoch 246
Validation binary_cross_entropy = 2.276274
Epoch 247
Validation binary_cross_entropy = 2.281345
Epoch 248
Validation binary_cross_entropy = 2.287288
Epoch 249
Loss = 1.0599e-05, PNorm = 92.8978, GNorm = 0.0005, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 2.292276
Epoch 250
Validation binary_cross_entropy = 2.297200
Epoch 251
Validation binary_cross_entropy = 2.299324
Epoch 252
Validation binary_cross_entropy = 2.301155
Epoch 253
Validation binary_cross_entropy = 2.301411
Epoch 254
Loss = 5.7398e-06, PNorm = 92.8997, GNorm = 0.0005, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 2.300556
Epoch 255
Validation binary_cross_entropy = 2.300749
Epoch 256
Validation binary_cross_entropy = 2.303829
Epoch 257
Validation binary_cross_entropy = 2.307087
Epoch 258
Validation binary_cross_entropy = 2.310578
Epoch 259
Loss = 4.5716e-06, PNorm = 92.9015, GNorm = 0.0002, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 2.313437
Epoch 260
Validation binary_cross_entropy = 2.314696
Epoch 261
Validation binary_cross_entropy = 2.318002
Epoch 262
Validation binary_cross_entropy = 2.323940
Epoch 263
Validation binary_cross_entropy = 2.327849
Epoch 264
Loss = 8.3380e-06, PNorm = 92.9030, GNorm = 0.0006, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 2.330935
Epoch 265
Validation binary_cross_entropy = 2.334590
Epoch 266
Validation binary_cross_entropy = 2.338541
Epoch 267
Validation binary_cross_entropy = 2.342999
Epoch 268
Validation binary_cross_entropy = 2.344758
Epoch 269
Loss = 1.7947e-05, PNorm = 92.9048, GNorm = 0.0004, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 2.349787
Epoch 270
Validation binary_cross_entropy = 2.356236
Epoch 271
Validation binary_cross_entropy = 2.364202
Epoch 272
Validation binary_cross_entropy = 2.370004
Epoch 273
Validation binary_cross_entropy = 2.375827
Epoch 274
Loss = 1.0472e-05, PNorm = 92.9078, GNorm = 0.0003, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 2.381230
Epoch 275
Validation binary_cross_entropy = 2.384420
Epoch 276
Validation binary_cross_entropy = 2.388703
Epoch 277
Validation binary_cross_entropy = 2.390203
Epoch 278
Validation binary_cross_entropy = 2.393411
Epoch 279
Loss = 5.6670e-06, PNorm = 92.9103, GNorm = 0.0003, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 2.394465
Epoch 280
Validation binary_cross_entropy = 2.395700
Epoch 281
Validation binary_cross_entropy = 2.398537
Epoch 282
Validation binary_cross_entropy = 2.402794
Epoch 283
Validation binary_cross_entropy = 2.405545
Epoch 284
Loss = 4.4643e-06, PNorm = 92.9122, GNorm = 0.0003, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 2.406659
Epoch 285
Validation binary_cross_entropy = 2.410830
Epoch 286
Validation binary_cross_entropy = 2.415474
Epoch 287
Validation binary_cross_entropy = 2.418952
Epoch 288
Validation binary_cross_entropy = 2.423604
Epoch 289
Loss = 6.9101e-06, PNorm = 92.9137, GNorm = 0.0003, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 2.427931
Epoch 290
Validation binary_cross_entropy = 2.430073
Epoch 291
Validation binary_cross_entropy = 2.434486
Epoch 292
Validation binary_cross_entropy = 2.440084
Epoch 293
Validation binary_cross_entropy = 2.446199
Epoch 294
Loss = 9.0562e-06, PNorm = 92.9157, GNorm = 0.0003, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 2.452407
Epoch 295
Validation binary_cross_entropy = 2.457152
Epoch 296
Validation binary_cross_entropy = 2.459818
Epoch 297
Validation binary_cross_entropy = 2.459881
Epoch 298
Validation binary_cross_entropy = 2.458441
Epoch 299
Loss = 8.0867e-06, PNorm = 92.9177, GNorm = 0.0003, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 2.457985
Model 0 best validation binary_cross_entropy = 0.260686 on epoch 38
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.207512
Ensemble test binary_cross_entropy = 0.207512
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1700, bias=False)
        (W_h): Linear(in_features=1700, out_features=1700, bias=False)
        (W_o): Linear(in_features=1833, out_features=1700, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=1700, out_features=1700, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=1700, out_features=1700, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.0, inplace=False)
    (7): Linear(in_features=1700, out_features=1, bias=True)
  )
)
Number of parameters = 12,042,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.617528
Epoch 1
Validation binary_cross_entropy = 0.643490
Epoch 2
Validation binary_cross_entropy = 0.694895
Epoch 3
Validation binary_cross_entropy = 0.754814
Epoch 4
Loss = 6.5068e-01, PNorm = 84.5266, GNorm = 0.5852, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.637030
Epoch 5
Validation binary_cross_entropy = 0.697410
Epoch 6
Validation binary_cross_entropy = 0.949206
Epoch 7
Validation binary_cross_entropy = 0.513984
Epoch 8
Validation binary_cross_entropy = 0.890598
Epoch 9
Loss = 6.5167e-01, PNorm = 84.8004, GNorm = 0.8753, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.557811
Epoch 10
Validation binary_cross_entropy = 0.758300
Epoch 11
Validation binary_cross_entropy = 0.474189
Epoch 12
Validation binary_cross_entropy = 0.441450
Epoch 13
Validation binary_cross_entropy = 0.536113
Epoch 14
Loss = 4.9499e-01, PNorm = 85.1877, GNorm = 0.8383, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.428763
Epoch 15
Validation binary_cross_entropy = 0.279967
Epoch 16
Validation binary_cross_entropy = 0.431882
Epoch 17
Validation binary_cross_entropy = 0.405995
Epoch 18
Validation binary_cross_entropy = 0.573730
Epoch 19
Loss = 4.1725e-01, PNorm = 85.5473, GNorm = 1.5965, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.237900
Epoch 20
Validation binary_cross_entropy = 0.704106
Epoch 21
Validation binary_cross_entropy = 0.300504
Epoch 22
Validation binary_cross_entropy = 0.369702
Epoch 23
Validation binary_cross_entropy = 0.541345
Epoch 24
Loss = 4.8776e-01, PNorm = 85.8308, GNorm = 1.2388, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.495881
Epoch 25
Validation binary_cross_entropy = 0.328375
Epoch 26
Validation binary_cross_entropy = 0.314529
Epoch 27
Validation binary_cross_entropy = 0.425518
Epoch 28
Validation binary_cross_entropy = 0.403745
Epoch 29
Loss = 3.7837e-01, PNorm = 86.1253, GNorm = 0.2872, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.384559
Epoch 30
Validation binary_cross_entropy = 0.447522
Epoch 31
Validation binary_cross_entropy = 0.369213
Epoch 32
Validation binary_cross_entropy = 0.379526
Epoch 33
Validation binary_cross_entropy = 0.327895
Epoch 34
Loss = 3.7134e-01, PNorm = 86.4329, GNorm = 0.5359, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.465166
Epoch 35
Validation binary_cross_entropy = 0.307009
Epoch 36
Validation binary_cross_entropy = 0.554545
Epoch 37
Validation binary_cross_entropy = 0.436598
Epoch 38
Validation binary_cross_entropy = 0.330145
Epoch 39
Loss = 3.0778e-01, PNorm = 86.7372, GNorm = 0.4566, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.480898
Epoch 40
Validation binary_cross_entropy = 0.397192
Epoch 41
Validation binary_cross_entropy = 0.394052
Epoch 42
Validation binary_cross_entropy = 0.470601
Epoch 43
Validation binary_cross_entropy = 0.360985
Epoch 44
Loss = 2.1577e-01, PNorm = 87.1184, GNorm = 0.4126, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.563161
Epoch 45
Validation binary_cross_entropy = 0.398817
Epoch 46
Validation binary_cross_entropy = 0.512266
Epoch 47
Validation binary_cross_entropy = 0.413698
Epoch 48
Validation binary_cross_entropy = 0.709463
Epoch 49
Loss = 1.8999e-01, PNorm = 87.5236, GNorm = 0.9462, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.514141
Epoch 50
Validation binary_cross_entropy = 1.426378
Epoch 51
Validation binary_cross_entropy = 0.418511
Epoch 52
Validation binary_cross_entropy = 0.405468
Epoch 53
Validation binary_cross_entropy = 0.644411
Epoch 54
Loss = 3.0892e-01, PNorm = 87.8693, GNorm = 0.7701, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.463941
Epoch 55
Validation binary_cross_entropy = 0.331937
Epoch 56
Validation binary_cross_entropy = 0.369493
Epoch 57
Validation binary_cross_entropy = 0.481393
Epoch 58
Validation binary_cross_entropy = 0.527813
Epoch 59
Loss = 2.5284e-01, PNorm = 88.3389, GNorm = 0.6050, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.450619
Epoch 60
Validation binary_cross_entropy = 0.459391
Epoch 61
Validation binary_cross_entropy = 0.506432
Epoch 62
Validation binary_cross_entropy = 0.504291
Epoch 63
Validation binary_cross_entropy = 0.511072
Epoch 64
Loss = 1.1739e-01, PNorm = 88.7373, GNorm = 0.4975, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.573228
Epoch 65
Validation binary_cross_entropy = 0.619179
Epoch 66
Validation binary_cross_entropy = 0.696685
Epoch 67
Validation binary_cross_entropy = 0.695956
Epoch 68
Validation binary_cross_entropy = 0.665368
Epoch 69
Loss = 1.6533e-01, PNorm = 89.0491, GNorm = 0.4692, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.657300
Epoch 70
Validation binary_cross_entropy = 0.627501
Epoch 71
Validation binary_cross_entropy = 0.618930
Epoch 72
Validation binary_cross_entropy = 0.643186
Epoch 73
Validation binary_cross_entropy = 0.690692
Epoch 74
Loss = 4.2172e-02, PNorm = 89.3216, GNorm = 0.3406, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.710647
Epoch 75
Validation binary_cross_entropy = 0.746860
Epoch 76
Validation binary_cross_entropy = 0.823884
Epoch 77
Validation binary_cross_entropy = 0.816974
Epoch 78
Validation binary_cross_entropy = 0.797465
Epoch 79
Loss = 7.3371e-02, PNorm = 89.5683, GNorm = 1.7459, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.848931
Epoch 80
Validation binary_cross_entropy = 0.894094
Epoch 81
Validation binary_cross_entropy = 0.887249
Epoch 82
Validation binary_cross_entropy = 0.897266
Epoch 83
Validation binary_cross_entropy = 0.920909
Epoch 84
Loss = 1.8475e-02, PNorm = 89.7653, GNorm = 0.2276, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.964275
Epoch 85
Validation binary_cross_entropy = 1.062897
Epoch 86
Validation binary_cross_entropy = 1.047008
Epoch 87
Validation binary_cross_entropy = 1.024135
Epoch 88
Validation binary_cross_entropy = 1.033519
Epoch 89
Loss = 5.6082e-02, PNorm = 89.9087, GNorm = 1.3815, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 1.014926
Epoch 90
Validation binary_cross_entropy = 0.927268
Epoch 91
Validation binary_cross_entropy = 0.942316
Epoch 92
Validation binary_cross_entropy = 1.058732
Epoch 93
Validation binary_cross_entropy = 1.059500
Epoch 94
Loss = 5.9699e-02, PNorm = 90.0393, GNorm = 0.8843, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 1.138542
Epoch 95
Validation binary_cross_entropy = 1.080250
Epoch 96
Validation binary_cross_entropy = 1.119565
Epoch 97
Validation binary_cross_entropy = 1.210891
Epoch 98
Validation binary_cross_entropy = 1.232285
Epoch 99
Loss = 3.8305e-02, PNorm = 90.1945, GNorm = 1.3072, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 1.299467
Epoch 100
Validation binary_cross_entropy = 1.212022
Epoch 101
Validation binary_cross_entropy = 1.197363
Epoch 102
Validation binary_cross_entropy = 1.117602
Epoch 103
Validation binary_cross_entropy = 1.063446
Epoch 104
Loss = 1.3445e-02, PNorm = 90.3729, GNorm = 0.2781, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 0.977226
Epoch 105
Validation binary_cross_entropy = 0.975240
Epoch 106
Validation binary_cross_entropy = 1.045492
Epoch 107
Validation binary_cross_entropy = 1.071769
Epoch 108
Validation binary_cross_entropy = 1.075245
Epoch 109
Loss = 3.5038e-03, PNorm = 90.5534, GNorm = 0.0629, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 1.087187
Epoch 110
Validation binary_cross_entropy = 1.128295
Epoch 111
Validation binary_cross_entropy = 1.242693
Epoch 112
Validation binary_cross_entropy = 1.216417
Epoch 113
Validation binary_cross_entropy = 1.318184
Epoch 114
Loss = 1.1496e-01, PNorm = 90.7285, GNorm = 0.2687, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 1.375449
Epoch 115
Validation binary_cross_entropy = 1.205249
Epoch 116
Validation binary_cross_entropy = 1.332692
Epoch 117
Validation binary_cross_entropy = 1.400958
Epoch 118
Validation binary_cross_entropy = 1.114101
Epoch 119
Loss = 1.5321e-01, PNorm = 90.9278, GNorm = 3.8333, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 1.060328
Epoch 120
Validation binary_cross_entropy = 0.729607
Epoch 121
Validation binary_cross_entropy = 0.709412
Epoch 122
Validation binary_cross_entropy = 0.828028
Epoch 123
Validation binary_cross_entropy = 0.745206
Epoch 124
Loss = 1.0595e-01, PNorm = 91.2645, GNorm = 1.2387, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.692976
Epoch 125
Validation binary_cross_entropy = 0.674167
Epoch 126
Validation binary_cross_entropy = 0.698827
Epoch 127
Validation binary_cross_entropy = 0.733869
Epoch 128
Validation binary_cross_entropy = 0.771352
Epoch 129
Loss = 1.2740e-02, PNorm = 91.5934, GNorm = 0.1184, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 0.800879
Epoch 130
Validation binary_cross_entropy = 0.834969
Epoch 131
Validation binary_cross_entropy = 0.917265
Epoch 132
Validation binary_cross_entropy = 0.899710
Epoch 133
Validation binary_cross_entropy = 0.867054
Epoch 134
Loss = 3.6571e-02, PNorm = 91.8160, GNorm = 0.9009, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 0.861934
Epoch 135
Validation binary_cross_entropy = 0.851823
Epoch 136
Validation binary_cross_entropy = 0.931017
Epoch 137
Validation binary_cross_entropy = 0.927084
Epoch 138
Validation binary_cross_entropy = 0.952829
Epoch 139
Loss = 3.5377e-02, PNorm = 91.9828, GNorm = 1.3027, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 0.998763
Epoch 140
Validation binary_cross_entropy = 1.061980
Epoch 141
Validation binary_cross_entropy = 1.118833
Epoch 142
Validation binary_cross_entropy = 1.118815
Epoch 143
Validation binary_cross_entropy = 1.121071
Epoch 144
Loss = 1.2532e-02, PNorm = 92.1081, GNorm = 0.6488, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.103521
Epoch 145
Validation binary_cross_entropy = 1.051535
Epoch 146
Validation binary_cross_entropy = 1.029901
Epoch 147
Validation binary_cross_entropy = 1.037729
Epoch 148
Validation binary_cross_entropy = 1.052934
Epoch 149
Loss = 7.5774e-02, PNorm = 92.2131, GNorm = 0.0526, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.105407
Epoch 150
Validation binary_cross_entropy = 1.178286
Epoch 151
Validation binary_cross_entropy = 1.161739
Epoch 152
Validation binary_cross_entropy = 1.126684
Epoch 153
Validation binary_cross_entropy = 1.117094
Epoch 154
Loss = 1.3713e-02, PNorm = 92.2926, GNorm = 1.1930, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 1.136323
Epoch 155
Validation binary_cross_entropy = 1.231210
Epoch 156
Validation binary_cross_entropy = 1.285907
Epoch 157
Validation binary_cross_entropy = 1.259203
Epoch 158
Validation binary_cross_entropy = 1.246029
Epoch 159
Loss = 1.3524e-02, PNorm = 92.3735, GNorm = 0.5997, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.243156
Epoch 160
Validation binary_cross_entropy = 1.241144
Epoch 161
Validation binary_cross_entropy = 1.237951
Epoch 162
Validation binary_cross_entropy = 1.233702
Epoch 163
Validation binary_cross_entropy = 1.244104
Epoch 164
Loss = 3.0609e-03, PNorm = 92.4529, GNorm = 0.1185, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.265347
Epoch 165
Validation binary_cross_entropy = 1.287928
Epoch 166
Validation binary_cross_entropy = 1.312821
Epoch 167
Validation binary_cross_entropy = 1.337284
Epoch 168
Validation binary_cross_entropy = 1.372748
Epoch 169
Loss = 3.4488e-03, PNorm = 92.5160, GNorm = 0.1863, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.418079
Epoch 170
Validation binary_cross_entropy = 1.420033
Epoch 171
Validation binary_cross_entropy = 1.397378
Epoch 172
Validation binary_cross_entropy = 1.385371
Epoch 173
Validation binary_cross_entropy = 1.389699
Epoch 174
Loss = 9.8638e-04, PNorm = 92.5612, GNorm = 0.0109, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.405046
Epoch 175
Validation binary_cross_entropy = 1.420820
Epoch 176
Validation binary_cross_entropy = 1.436735
Epoch 177
Validation binary_cross_entropy = 1.454474
Epoch 178
Validation binary_cross_entropy = 1.479617
Epoch 179
Loss = 8.2774e-04, PNorm = 92.6027, GNorm = 0.0208, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.534159
Epoch 180
Validation binary_cross_entropy = 1.565942
Epoch 181
Validation binary_cross_entropy = 1.516040
Epoch 182
Validation binary_cross_entropy = 1.487898
Epoch 183
Validation binary_cross_entropy = 1.484294
Epoch 184
Loss = 4.0164e-04, PNorm = 92.6426, GNorm = 0.0066, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.493201
Epoch 185
Validation binary_cross_entropy = 1.509194
Epoch 186
Validation binary_cross_entropy = 1.541358
Epoch 187
Validation binary_cross_entropy = 1.620570
Epoch 188
Validation binary_cross_entropy = 1.714866
Epoch 189
Loss = 9.8233e-03, PNorm = 92.6885, GNorm = 0.8726, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 1.698961
Epoch 190
Validation binary_cross_entropy = 1.622903
Epoch 191
Validation binary_cross_entropy = 1.586513
Epoch 192
Validation binary_cross_entropy = 1.577307
Epoch 193
Validation binary_cross_entropy = 1.577030
Epoch 194
Loss = 5.4760e-04, PNorm = 92.7207, GNorm = 0.0778, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 1.581009
Epoch 195
Validation binary_cross_entropy = 1.587475
Epoch 196
Validation binary_cross_entropy = 1.593542
Epoch 197
Validation binary_cross_entropy = 1.598886
Epoch 198
Validation binary_cross_entropy = 1.603487
Epoch 199
Loss = 7.8124e-03, PNorm = 92.7441, GNorm = 0.0203, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 1.602176
Epoch 200
Validation binary_cross_entropy = 1.611138
Epoch 201
Validation binary_cross_entropy = 1.633448
Epoch 202
Validation binary_cross_entropy = 1.641102
Epoch 203
Validation binary_cross_entropy = 1.641246
Epoch 204
Loss = 1.1342e-03, PNorm = 92.7703, GNorm = 0.0338, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.629372
Epoch 205
Validation binary_cross_entropy = 1.616450
Epoch 206
Validation binary_cross_entropy = 1.604426
Epoch 207
Validation binary_cross_entropy = 1.595429
Epoch 208
Validation binary_cross_entropy = 1.588256
Epoch 209
Loss = 2.0068e-04, PNorm = 92.7920, GNorm = 0.0033, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.582340
Epoch 210
Validation binary_cross_entropy = 1.591525
Epoch 211
Validation binary_cross_entropy = 1.627906
Epoch 212
Validation binary_cross_entropy = 1.659636
Epoch 213
Validation binary_cross_entropy = 1.686743
Epoch 214
Loss = 2.9782e-04, PNorm = 92.8092, GNorm = 0.0141, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.708416
Epoch 215
Validation binary_cross_entropy = 1.724118
Epoch 216
Validation binary_cross_entropy = 1.732939
Epoch 217
Validation binary_cross_entropy = 1.735236
Epoch 218
Validation binary_cross_entropy = 1.732059
Epoch 219
Loss = 1.9949e-04, PNorm = 92.8239, GNorm = 0.0055, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.723432
Epoch 220
Validation binary_cross_entropy = 1.714976
Epoch 221
Validation binary_cross_entropy = 1.708930
Epoch 222
Validation binary_cross_entropy = 1.730584
Epoch 223
Validation binary_cross_entropy = 1.846700
Epoch 224
Loss = 1.0358e-02, PNorm = 92.8419, GNorm = 0.0601, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.728504
Epoch 225
Validation binary_cross_entropy = 1.679065
Epoch 226
Validation binary_cross_entropy = 1.667186
Epoch 227
Validation binary_cross_entropy = 1.666180
Epoch 228
Validation binary_cross_entropy = 1.666726
Epoch 229
Loss = 3.3513e-04, PNorm = 92.8665, GNorm = 0.0117, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.670642
Epoch 230
Validation binary_cross_entropy = 1.706119
Epoch 231
Validation binary_cross_entropy = 1.829413
Epoch 232
Validation binary_cross_entropy = 1.994236
Epoch 233
Validation binary_cross_entropy = 1.772021
Epoch 234
Loss = 1.2389e-03, PNorm = 92.9177, GNorm = 0.1505, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.646635
Epoch 235
Validation binary_cross_entropy = 1.568229
Epoch 236
Validation binary_cross_entropy = 1.398872
Epoch 237
Validation binary_cross_entropy = 1.312549
Epoch 238
Validation binary_cross_entropy = 1.327988
Epoch 239
Loss = 1.8411e-02, PNorm = 93.0083, GNorm = 1.6885, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.317373
Epoch 240
Validation binary_cross_entropy = 1.227319
Epoch 241
Validation binary_cross_entropy = 1.183250
Epoch 242
Validation binary_cross_entropy = 1.180966
Epoch 243
Validation binary_cross_entropy = 1.192631
Epoch 244
Loss = 4.5981e-03, PNorm = 93.1063, GNorm = 0.1184, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.197323
Epoch 245
Validation binary_cross_entropy = 1.200006
Epoch 246
Validation binary_cross_entropy = 1.207134
Epoch 247
Validation binary_cross_entropy = 1.290168
Epoch 248
Validation binary_cross_entropy = 1.336914
Epoch 249
Loss = 1.2213e-02, PNorm = 93.1751, GNorm = 1.1057, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.325310
Epoch 250
Validation binary_cross_entropy = 1.307616
Epoch 251
Validation binary_cross_entropy = 1.309897
Epoch 252
Validation binary_cross_entropy = 1.324518
Epoch 253
Validation binary_cross_entropy = 1.352247
Epoch 254
Loss = 6.5087e-04, PNorm = 93.2186, GNorm = 0.0023, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.410222
Epoch 255
Validation binary_cross_entropy = 1.473746
Epoch 256
Validation binary_cross_entropy = 1.534148
Epoch 257
Validation binary_cross_entropy = 1.580519
Epoch 258
Validation binary_cross_entropy = 1.609273
Epoch 259
Loss = 4.9932e-02, PNorm = 93.2576, GNorm = 1.4404, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.694637
Epoch 260
Validation binary_cross_entropy = 1.794356
Epoch 261
Validation binary_cross_entropy = 1.735821
Epoch 262
Validation binary_cross_entropy = 1.644631
Epoch 263
Validation binary_cross_entropy = 1.613784
Epoch 264
Loss = 1.9080e-04, PNorm = 93.3095, GNorm = 0.0018, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.621001
Epoch 265
Validation binary_cross_entropy = 1.639809
Epoch 266
Validation binary_cross_entropy = 1.653751
Epoch 267
Validation binary_cross_entropy = 1.667428
Epoch 268
Validation binary_cross_entropy = 1.678382
Epoch 269
Loss = 1.6398e-04, PNorm = 93.3536, GNorm = 0.0116, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.688620
Epoch 270
Validation binary_cross_entropy = 1.691920
Epoch 271
Validation binary_cross_entropy = 1.697809
Epoch 272
Validation binary_cross_entropy = 1.705552
Epoch 273
Validation binary_cross_entropy = 1.714290
Epoch 274
Loss = 1.8473e-04, PNorm = 93.3796, GNorm = 0.0082, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.722924
Epoch 275
Validation binary_cross_entropy = 1.732437
Epoch 276
Validation binary_cross_entropy = 1.741650
Epoch 277
Validation binary_cross_entropy = 1.749907
Epoch 278
Validation binary_cross_entropy = 1.756268
Epoch 279
Loss = 2.0355e-04, PNorm = 93.3924, GNorm = 0.0057, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.761084
Epoch 280
Validation binary_cross_entropy = 1.764540
Epoch 281
Validation binary_cross_entropy = 1.767545
Epoch 282
Validation binary_cross_entropy = 1.769449
Epoch 283
Validation binary_cross_entropy = 1.770748
Epoch 284
Loss = 1.4335e-04, PNorm = 93.3994, GNorm = 0.0018, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.771716
Epoch 285
Validation binary_cross_entropy = 1.772295
Epoch 286
Validation binary_cross_entropy = 1.772913
Epoch 287
Validation binary_cross_entropy = 1.774818
Epoch 288
Validation binary_cross_entropy = 1.780389
Epoch 289
Loss = 5.9201e-04, PNorm = 93.4058, GNorm = 0.0015, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.781137
Epoch 290
Validation binary_cross_entropy = 1.783166
Epoch 291
Validation binary_cross_entropy = 1.785783
Epoch 292
Validation binary_cross_entropy = 1.788236
Epoch 293
Validation binary_cross_entropy = 1.798604
Epoch 294
Loss = 1.6703e-04, PNorm = 93.4113, GNorm = 0.0030, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.810875
Epoch 295
Validation binary_cross_entropy = 1.821184
Epoch 296
Validation binary_cross_entropy = 1.829464
Epoch 297
Validation binary_cross_entropy = 1.835098
Epoch 298
Validation binary_cross_entropy = 1.838835
Epoch 299
Loss = 1.6671e-04, PNorm = 93.4170, GNorm = 0.0038, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.840193
Model 0 best validation binary_cross_entropy = 0.237900 on epoch 19
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.187627
Ensemble test binary_cross_entropy = 0.187627
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.0, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1700, bias=False)
        (W_h): Linear(in_features=1700, out_features=1700, bias=False)
        (W_o): Linear(in_features=1833, out_features=1700, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.0, inplace=False)
    (1): Linear(in_features=1700, out_features=1700, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.0, inplace=False)
    (4): Linear(in_features=1700, out_features=1700, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.0, inplace=False)
    (7): Linear(in_features=1700, out_features=1, bias=True)
  )
)
Number of parameters = 12,042,801
Moving model to cuda
Epoch 0
Validation binary_cross_entropy = 0.623308
Epoch 1
Validation binary_cross_entropy = 0.626315
Epoch 2
Validation binary_cross_entropy = 0.737243
Epoch 3
Validation binary_cross_entropy = 0.626868
Epoch 4
Loss = 6.6188e-01, PNorm = 84.5207, GNorm = 0.2935, lr_0 = 6.5000e-04
Validation binary_cross_entropy = 0.712586
Epoch 5
Validation binary_cross_entropy = 0.989161
Epoch 6
Validation binary_cross_entropy = 0.549920
Epoch 7
Validation binary_cross_entropy = 0.750689
Epoch 8
Validation binary_cross_entropy = 0.683380
Epoch 9
Loss = 6.3560e-01, PNorm = 84.7780, GNorm = 0.3616, lr_0 = 9.9743e-04
Validation binary_cross_entropy = 0.539476
Epoch 10
Validation binary_cross_entropy = 0.772628
Epoch 11
Validation binary_cross_entropy = 0.363309
Epoch 12
Validation binary_cross_entropy = 0.661060
Epoch 13
Validation binary_cross_entropy = 0.380016
Epoch 14
Loss = 4.9708e-01, PNorm = 85.1732, GNorm = 1.1881, lr_0 = 9.8890e-04
Validation binary_cross_entropy = 0.924879
Epoch 15
Validation binary_cross_entropy = 0.270928
Epoch 16
Validation binary_cross_entropy = 0.521089
Epoch 17
Validation binary_cross_entropy = 0.736534
Epoch 18
Validation binary_cross_entropy = 0.541982
Epoch 19
Loss = 4.7905e-01, PNorm = 85.5479, GNorm = 0.4771, lr_0 = 9.8045e-04
Validation binary_cross_entropy = 0.437496
Epoch 20
Validation binary_cross_entropy = 0.415878
Epoch 21
Validation binary_cross_entropy = 0.426908
Epoch 22
Validation binary_cross_entropy = 0.307551
Epoch 23
Validation binary_cross_entropy = 0.398279
Epoch 24
Loss = 4.0323e-01, PNorm = 85.9645, GNorm = 0.5819, lr_0 = 9.7207e-04
Validation binary_cross_entropy = 0.518815
Epoch 25
Validation binary_cross_entropy = 0.326639
Epoch 26
Validation binary_cross_entropy = 0.399101
Epoch 27
Validation binary_cross_entropy = 0.424100
Epoch 28
Validation binary_cross_entropy = 0.370427
Epoch 29
Loss = 3.7255e-01, PNorm = 86.3663, GNorm = 1.4571, lr_0 = 9.6376e-04
Validation binary_cross_entropy = 0.308674
Epoch 30
Validation binary_cross_entropy = 0.344283
Epoch 31
Validation binary_cross_entropy = 0.462877
Epoch 32
Validation binary_cross_entropy = 0.333520
Epoch 33
Validation binary_cross_entropy = 0.324690
Epoch 34
Loss = 2.9727e-01, PNorm = 86.7320, GNorm = 0.7794, lr_0 = 9.5552e-04
Validation binary_cross_entropy = 0.421604
Epoch 35
Validation binary_cross_entropy = 0.290265
Epoch 36
Validation binary_cross_entropy = 0.358445
Epoch 37
Validation binary_cross_entropy = 0.362074
Epoch 38
Validation binary_cross_entropy = 0.468651
Epoch 39
Loss = 3.1598e-01, PNorm = 87.1130, GNorm = 0.8213, lr_0 = 9.4735e-04
Validation binary_cross_entropy = 0.387578
Epoch 40
Validation binary_cross_entropy = 0.368574
Epoch 41
Validation binary_cross_entropy = 0.414210
Epoch 42
Validation binary_cross_entropy = 0.416054
Epoch 43
Validation binary_cross_entropy = 0.479965
Epoch 44
Loss = 2.6016e-01, PNorm = 87.5380, GNorm = 1.0746, lr_0 = 9.3925e-04
Validation binary_cross_entropy = 0.575312
Epoch 45
Validation binary_cross_entropy = 0.616939
Epoch 46
Validation binary_cross_entropy = 0.503306
Epoch 47
Validation binary_cross_entropy = 0.996396
Epoch 48
Validation binary_cross_entropy = 0.520320
Epoch 49
Loss = 3.0706e-01, PNorm = 87.9204, GNorm = 1.5057, lr_0 = 9.3122e-04
Validation binary_cross_entropy = 0.380041
Epoch 50
Validation binary_cross_entropy = 0.375915
Epoch 51
Validation binary_cross_entropy = 0.371494
Epoch 52
Validation binary_cross_entropy = 0.460734
Epoch 53
Validation binary_cross_entropy = 0.446257
Epoch 54
Loss = 1.5594e-01, PNorm = 88.3696, GNorm = 0.9151, lr_0 = 9.2326e-04
Validation binary_cross_entropy = 0.519944
Epoch 55
Validation binary_cross_entropy = 0.603667
Epoch 56
Validation binary_cross_entropy = 0.605838
Epoch 57
Validation binary_cross_entropy = 0.573969
Epoch 58
Validation binary_cross_entropy = 0.505636
Epoch 59
Loss = 1.5732e-01, PNorm = 88.7419, GNorm = 1.4437, lr_0 = 9.1537e-04
Validation binary_cross_entropy = 0.511662
Epoch 60
Validation binary_cross_entropy = 0.511805
Epoch 61
Validation binary_cross_entropy = 0.535639
Epoch 62
Validation binary_cross_entropy = 0.557527
Epoch 63
Validation binary_cross_entropy = 0.643793
Epoch 64
Loss = 1.1509e-01, PNorm = 89.1012, GNorm = 1.9350, lr_0 = 9.0754e-04
Validation binary_cross_entropy = 0.743410
Epoch 65
Validation binary_cross_entropy = 0.825211
Epoch 66
Validation binary_cross_entropy = 0.883657
Epoch 67
Validation binary_cross_entropy = 0.888129
Epoch 68
Validation binary_cross_entropy = 0.775346
Epoch 69
Loss = 1.7648e-02, PNorm = 89.3723, GNorm = 0.6517, lr_0 = 8.9978e-04
Validation binary_cross_entropy = 0.749265
Epoch 70
Validation binary_cross_entropy = 0.791745
Epoch 71
Validation binary_cross_entropy = 0.816197
Epoch 72
Validation binary_cross_entropy = 0.944061
Epoch 73
Validation binary_cross_entropy = 0.961296
Epoch 74
Loss = 2.2753e-01, PNorm = 89.5794, GNorm = 0.3961, lr_0 = 8.9209e-04
Validation binary_cross_entropy = 0.893711
Epoch 75
Validation binary_cross_entropy = 0.549706
Epoch 76
Validation binary_cross_entropy = 0.502334
Epoch 77
Validation binary_cross_entropy = 0.701081
Epoch 78
Validation binary_cross_entropy = 0.516566
Epoch 79
Loss = 1.1522e-01, PNorm = 89.8830, GNorm = 0.4479, lr_0 = 8.8447e-04
Validation binary_cross_entropy = 0.443223
Epoch 80
Validation binary_cross_entropy = 0.476379
Epoch 81
Validation binary_cross_entropy = 0.502109
Epoch 82
Validation binary_cross_entropy = 0.532667
Epoch 83
Validation binary_cross_entropy = 0.578687
Epoch 84
Loss = 8.7566e-02, PNorm = 90.2156, GNorm = 0.4333, lr_0 = 8.7691e-04
Validation binary_cross_entropy = 0.624649
Epoch 85
Validation binary_cross_entropy = 0.653505
Epoch 86
Validation binary_cross_entropy = 0.690964
Epoch 87
Validation binary_cross_entropy = 0.728685
Epoch 88
Validation binary_cross_entropy = 0.751567
Epoch 89
Loss = 2.6348e-02, PNorm = 90.4525, GNorm = 0.2090, lr_0 = 8.6941e-04
Validation binary_cross_entropy = 0.813643
Epoch 90
Validation binary_cross_entropy = 0.897768
Epoch 91
Validation binary_cross_entropy = 0.926518
Epoch 92
Validation binary_cross_entropy = 0.863111
Epoch 93
Validation binary_cross_entropy = 0.853375
Epoch 94
Loss = 3.0026e-02, PNorm = 90.6458, GNorm = 0.4556, lr_0 = 8.6198e-04
Validation binary_cross_entropy = 0.855241
Epoch 95
Validation binary_cross_entropy = 0.855257
Epoch 96
Validation binary_cross_entropy = 0.856445
Epoch 97
Validation binary_cross_entropy = 0.873777
Epoch 98
Validation binary_cross_entropy = 0.901985
Epoch 99
Loss = 7.5522e-02, PNorm = 90.8175, GNorm = 0.5355, lr_0 = 8.5461e-04
Validation binary_cross_entropy = 0.936653
Epoch 100
Validation binary_cross_entropy = 0.975151
Epoch 101
Validation binary_cross_entropy = 1.027412
Epoch 102
Validation binary_cross_entropy = 1.063061
Epoch 103
Validation binary_cross_entropy = 1.100415
Epoch 104
Loss = 3.3445e-02, PNorm = 90.9304, GNorm = 0.9011, lr_0 = 8.4730e-04
Validation binary_cross_entropy = 1.176663
Epoch 105
Validation binary_cross_entropy = 1.192063
Epoch 106
Validation binary_cross_entropy = 1.141423
Epoch 107
Validation binary_cross_entropy = 1.133988
Epoch 108
Validation binary_cross_entropy = 1.143123
Epoch 109
Loss = 1.8211e-03, PNorm = 91.0233, GNorm = 0.0306, lr_0 = 8.4006e-04
Validation binary_cross_entropy = 1.168845
Epoch 110
Validation binary_cross_entropy = 1.192099
Epoch 111
Validation binary_cross_entropy = 1.201414
Epoch 112
Validation binary_cross_entropy = 1.187157
Epoch 113
Validation binary_cross_entropy = 1.211850
Epoch 114
Loss = 4.3433e-02, PNorm = 91.1049, GNorm = 0.2300, lr_0 = 8.3288e-04
Validation binary_cross_entropy = 1.242322
Epoch 115
Validation binary_cross_entropy = 1.281868
Epoch 116
Validation binary_cross_entropy = 1.136405
Epoch 117
Validation binary_cross_entropy = 1.264247
Epoch 118
Validation binary_cross_entropy = 1.101024
Epoch 119
Loss = 3.7989e-03, PNorm = 91.1914, GNorm = 0.1800, lr_0 = 8.2576e-04
Validation binary_cross_entropy = 1.069598
Epoch 120
Validation binary_cross_entropy = 1.026241
Epoch 121
Validation binary_cross_entropy = 0.941609
Epoch 122
Validation binary_cross_entropy = 0.898969
Epoch 123
Validation binary_cross_entropy = 0.904703
Epoch 124
Loss = 4.7015e-02, PNorm = 91.2933, GNorm = 0.2972, lr_0 = 8.1870e-04
Validation binary_cross_entropy = 0.921278
Epoch 125
Validation binary_cross_entropy = 0.933863
Epoch 126
Validation binary_cross_entropy = 0.971340
Epoch 127
Validation binary_cross_entropy = 1.014133
Epoch 128
Validation binary_cross_entropy = 1.096687
Epoch 129
Loss = 6.3372e-03, PNorm = 91.4084, GNorm = 0.1850, lr_0 = 8.1170e-04
Validation binary_cross_entropy = 1.146981
Epoch 130
Validation binary_cross_entropy = 1.193402
Epoch 131
Validation binary_cross_entropy = 1.248194
Epoch 132
Validation binary_cross_entropy = 1.308933
Epoch 133
Validation binary_cross_entropy = 1.335027
Epoch 134
Loss = 2.4262e-03, PNorm = 91.5066, GNorm = 0.0749, lr_0 = 8.0476e-04
Validation binary_cross_entropy = 1.345625
Epoch 135
Validation binary_cross_entropy = 1.365392
Epoch 136
Validation binary_cross_entropy = 1.389064
Epoch 137
Validation binary_cross_entropy = 1.415478
Epoch 138
Validation binary_cross_entropy = 1.441976
Epoch 139
Loss = 4.4551e-04, PNorm = 91.5668, GNorm = 0.0214, lr_0 = 7.9788e-04
Validation binary_cross_entropy = 1.466071
Epoch 140
Validation binary_cross_entropy = 1.494340
Epoch 141
Validation binary_cross_entropy = 1.530460
Epoch 142
Validation binary_cross_entropy = 1.495057
Epoch 143
Validation binary_cross_entropy = 1.474573
Epoch 144
Loss = 1.3299e-03, PNorm = 91.5987, GNorm = 0.1634, lr_0 = 7.9106e-04
Validation binary_cross_entropy = 1.474795
Epoch 145
Validation binary_cross_entropy = 1.549012
Epoch 146
Validation binary_cross_entropy = 1.583673
Epoch 147
Validation binary_cross_entropy = 1.586101
Epoch 148
Validation binary_cross_entropy = 1.639704
Epoch 149
Loss = 6.4099e-03, PNorm = 91.6685, GNorm = 0.2027, lr_0 = 7.8430e-04
Validation binary_cross_entropy = 1.542027
Epoch 150
Validation binary_cross_entropy = 1.473803
Epoch 151
Validation binary_cross_entropy = 1.458951
Epoch 152
Validation binary_cross_entropy = 1.409073
Epoch 153
Validation binary_cross_entropy = 1.399139
Epoch 154
Loss = 1.5530e-02, PNorm = 91.7630, GNorm = 1.8791, lr_0 = 7.7759e-04
Validation binary_cross_entropy = 1.339303
Epoch 155
Validation binary_cross_entropy = 1.366642
Epoch 156
Validation binary_cross_entropy = 1.334941
Epoch 157
Validation binary_cross_entropy = 1.396089
Epoch 158
Validation binary_cross_entropy = 1.474944
Epoch 159
Loss = 3.0467e-03, PNorm = 91.8456, GNorm = 0.1319, lr_0 = 7.7095e-04
Validation binary_cross_entropy = 1.516505
Epoch 160
Validation binary_cross_entropy = 1.498613
Epoch 161
Validation binary_cross_entropy = 1.522448
Epoch 162
Validation binary_cross_entropy = 1.566903
Epoch 163
Validation binary_cross_entropy = 1.597690
Epoch 164
Loss = 6.4976e-03, PNorm = 91.9499, GNorm = 1.2017, lr_0 = 7.6436e-04
Validation binary_cross_entropy = 1.556970
Epoch 165
Validation binary_cross_entropy = 1.561771
Epoch 166
Validation binary_cross_entropy = 1.493998
Epoch 167
Validation binary_cross_entropy = 1.448194
Epoch 168
Validation binary_cross_entropy = 1.540082
Epoch 169
Loss = 6.9077e-02, PNorm = 92.0631, GNorm = 3.1904, lr_0 = 7.5782e-04
Validation binary_cross_entropy = 1.470541
Epoch 170
Validation binary_cross_entropy = 1.310409
Epoch 171
Validation binary_cross_entropy = 1.294425
Epoch 172
Validation binary_cross_entropy = 1.219399
Epoch 173
Validation binary_cross_entropy = 1.188503
Epoch 174
Loss = 6.1432e-02, PNorm = 92.2182, GNorm = 2.3978, lr_0 = 7.5134e-04
Validation binary_cross_entropy = 1.190456
Epoch 175
Validation binary_cross_entropy = 1.182396
Epoch 176
Validation binary_cross_entropy = 1.320503
Epoch 177
Validation binary_cross_entropy = 1.390321
Epoch 178
Validation binary_cross_entropy = 1.460919
Epoch 179
Loss = 6.5148e-03, PNorm = 92.4027, GNorm = 0.5429, lr_0 = 7.4492e-04
Validation binary_cross_entropy = 1.493640
Epoch 180
Validation binary_cross_entropy = 1.492557
Epoch 181
Validation binary_cross_entropy = 1.484804
Epoch 182
Validation binary_cross_entropy = 1.427718
Epoch 183
Validation binary_cross_entropy = 1.527742
Epoch 184
Loss = 1.6593e-01, PNorm = 92.5734, GNorm = 6.0975, lr_0 = 7.3855e-04
Validation binary_cross_entropy = 1.444992
Epoch 185
Validation binary_cross_entropy = 1.184087
Epoch 186
Validation binary_cross_entropy = 1.614812
Epoch 187
Validation binary_cross_entropy = 0.861211
Epoch 188
Validation binary_cross_entropy = 0.585403
Epoch 189
Loss = 2.1804e-01, PNorm = 92.8582, GNorm = 2.4262, lr_0 = 7.3224e-04
Validation binary_cross_entropy = 0.572485
Epoch 190
Validation binary_cross_entropy = 0.443423
Epoch 191
Validation binary_cross_entropy = 0.458400
Epoch 192
Validation binary_cross_entropy = 0.495049
Epoch 193
Validation binary_cross_entropy = 0.605015
Epoch 194
Loss = 5.8143e-02, PNorm = 93.3065, GNorm = 0.5893, lr_0 = 7.2598e-04
Validation binary_cross_entropy = 0.642984
Epoch 195
Validation binary_cross_entropy = 0.708747
Epoch 196
Validation binary_cross_entropy = 0.818980
Epoch 197
Validation binary_cross_entropy = 0.905240
Epoch 198
Validation binary_cross_entropy = 0.947679
Epoch 199
Loss = 7.3894e-02, PNorm = 93.6554, GNorm = 0.4524, lr_0 = 7.1977e-04
Validation binary_cross_entropy = 0.938772
Epoch 200
Validation binary_cross_entropy = 0.952934
Epoch 201
Validation binary_cross_entropy = 0.973858
Epoch 202
Validation binary_cross_entropy = 0.995495
Epoch 203
Validation binary_cross_entropy = 1.009279
Epoch 204
Loss = 6.0209e-02, PNorm = 93.8555, GNorm = 0.1593, lr_0 = 7.1362e-04
Validation binary_cross_entropy = 1.041823
Epoch 205
Validation binary_cross_entropy = 1.078744
Epoch 206
Validation binary_cross_entropy = 1.085551
Epoch 207
Validation binary_cross_entropy = 1.091384
Epoch 208
Validation binary_cross_entropy = 1.106438
Epoch 209
Loss = 4.4683e-03, PNorm = 93.9822, GNorm = 0.0750, lr_0 = 7.0752e-04
Validation binary_cross_entropy = 1.130399
Epoch 210
Validation binary_cross_entropy = 1.160001
Epoch 211
Validation binary_cross_entropy = 1.190349
Epoch 212
Validation binary_cross_entropy = 1.214100
Epoch 213
Validation binary_cross_entropy = 1.240904
Epoch 214
Loss = 9.1719e-03, PNorm = 94.0789, GNorm = 0.2918, lr_0 = 7.0147e-04
Validation binary_cross_entropy = 1.313278
Epoch 215
Validation binary_cross_entropy = 1.368039
Epoch 216
Validation binary_cross_entropy = 1.378943
Epoch 217
Validation binary_cross_entropy = 1.370934
Epoch 218
Validation binary_cross_entropy = 1.373272
Epoch 219
Loss = 7.5654e-03, PNorm = 94.1545, GNorm = 0.3696, lr_0 = 6.9548e-04
Validation binary_cross_entropy = 1.425414
Epoch 220
Validation binary_cross_entropy = 1.474546
Epoch 221
Validation binary_cross_entropy = 1.504484
Epoch 222
Validation binary_cross_entropy = 1.515069
Epoch 223
Validation binary_cross_entropy = 1.519776
Epoch 224
Loss = 3.0687e-02, PNorm = 94.2215, GNorm = 0.1439, lr_0 = 6.8953e-04
Validation binary_cross_entropy = 1.566345
Epoch 225
Validation binary_cross_entropy = 1.562262
Epoch 226
Validation binary_cross_entropy = 1.526898
Epoch 227
Validation binary_cross_entropy = 1.498364
Epoch 228
Validation binary_cross_entropy = 1.495107
Epoch 229
Loss = 1.2948e-03, PNorm = 94.2829, GNorm = 0.1662, lr_0 = 6.8364e-04
Validation binary_cross_entropy = 1.506634
Epoch 230
Validation binary_cross_entropy = 1.528906
Epoch 231
Validation binary_cross_entropy = 1.565062
Epoch 232
Validation binary_cross_entropy = 1.611388
Epoch 233
Validation binary_cross_entropy = 1.652955
Epoch 234
Loss = 6.5321e-03, PNorm = 94.3414, GNorm = 0.4232, lr_0 = 6.7779e-04
Validation binary_cross_entropy = 1.662330
Epoch 235
Validation binary_cross_entropy = 1.635793
Epoch 236
Validation binary_cross_entropy = 1.612012
Epoch 237
Validation binary_cross_entropy = 1.591617
Epoch 238
Validation binary_cross_entropy = 1.580780
Epoch 239
Loss = 1.2718e-03, PNorm = 94.3777, GNorm = 0.0932, lr_0 = 6.7200e-04
Validation binary_cross_entropy = 1.581534
Epoch 240
Validation binary_cross_entropy = 1.590191
Epoch 241
Validation binary_cross_entropy = 1.618723
Epoch 242
Validation binary_cross_entropy = 1.687500
Epoch 243
Validation binary_cross_entropy = 1.778084
Epoch 244
Loss = 4.0971e-03, PNorm = 94.4129, GNorm = 0.2498, lr_0 = 6.6625e-04
Validation binary_cross_entropy = 1.837854
Epoch 245
Validation binary_cross_entropy = 1.776060
Epoch 246
Validation binary_cross_entropy = 1.733581
Epoch 247
Validation binary_cross_entropy = 1.721029
Epoch 248
Validation binary_cross_entropy = 1.726011
Epoch 249
Loss = 1.6821e-03, PNorm = 94.4451, GNorm = 0.1557, lr_0 = 6.6056e-04
Validation binary_cross_entropy = 1.739838
Epoch 250
Validation binary_cross_entropy = 1.876197
Epoch 251
Validation binary_cross_entropy = 1.673513
Epoch 252
Validation binary_cross_entropy = 1.624806
Epoch 253
Validation binary_cross_entropy = 1.426219
Epoch 254
Loss = 9.0070e-03, PNorm = 94.5577, GNorm = 0.0027, lr_0 = 6.5491e-04
Validation binary_cross_entropy = 1.370528
Epoch 255
Validation binary_cross_entropy = 1.477117
Epoch 256
Validation binary_cross_entropy = 1.329000
Epoch 257
Validation binary_cross_entropy = 1.262093
Epoch 258
Validation binary_cross_entropy = 1.152821
Epoch 259
Loss = 1.2031e-02, PNorm = 94.7090, GNorm = 0.1286, lr_0 = 6.4931e-04
Validation binary_cross_entropy = 1.088298
Epoch 260
Validation binary_cross_entropy = 1.042595
Epoch 261
Validation binary_cross_entropy = 1.057758
Epoch 262
Validation binary_cross_entropy = 1.070069
Epoch 263
Validation binary_cross_entropy = 1.036117
Epoch 264
Loss = 6.7018e-03, PNorm = 94.8493, GNorm = 0.0875, lr_0 = 6.4376e-04
Validation binary_cross_entropy = 1.028537
Epoch 265
Validation binary_cross_entropy = 1.045049
Epoch 266
Validation binary_cross_entropy = 1.091009
Epoch 267
Validation binary_cross_entropy = 1.147385
Epoch 268
Validation binary_cross_entropy = 1.193173
Epoch 269
Loss = 3.0164e-02, PNorm = 94.9670, GNorm = 1.7644, lr_0 = 6.3826e-04
Validation binary_cross_entropy = 1.229669
Epoch 270
Validation binary_cross_entropy = 1.268655
Epoch 271
Validation binary_cross_entropy = 1.296181
Epoch 272
Validation binary_cross_entropy = 1.312595
Epoch 273
Validation binary_cross_entropy = 1.324200
Epoch 274
Loss = 1.3965e-03, PNorm = 95.0508, GNorm = 0.0169, lr_0 = 6.3280e-04
Validation binary_cross_entropy = 1.337412
Epoch 275
Validation binary_cross_entropy = 1.355469
Epoch 276
Validation binary_cross_entropy = 1.394169
Epoch 277
Validation binary_cross_entropy = 1.428744
Epoch 278
Validation binary_cross_entropy = 1.453548
Epoch 279
Loss = 1.4214e-03, PNorm = 95.1091, GNorm = 0.1105, lr_0 = 6.2739e-04
Validation binary_cross_entropy = 1.469141
Epoch 280
Validation binary_cross_entropy = 1.482381
Epoch 281
Validation binary_cross_entropy = 1.494245
Epoch 282
Validation binary_cross_entropy = 1.504834
Epoch 283
Validation binary_cross_entropy = 1.517065
Epoch 284
Loss = 2.7176e-04, PNorm = 95.1461, GNorm = 0.0053, lr_0 = 6.2203e-04
Validation binary_cross_entropy = 1.527640
Epoch 285
Validation binary_cross_entropy = 1.537165
Epoch 286
Validation binary_cross_entropy = 1.547169
Epoch 287
Validation binary_cross_entropy = 1.554274
Epoch 288
Validation binary_cross_entropy = 1.561230
Epoch 289
Loss = 1.3747e-04, PNorm = 95.1699, GNorm = 0.0040, lr_0 = 6.1671e-04
Validation binary_cross_entropy = 1.567766
Epoch 290
Validation binary_cross_entropy = 1.578043
Epoch 291
Validation binary_cross_entropy = 1.583274
Epoch 292
Validation binary_cross_entropy = 1.588915
Epoch 293
Validation binary_cross_entropy = 1.594136
Epoch 294
Loss = 2.7020e-04, PNorm = 95.1856, GNorm = 0.0039, lr_0 = 6.1144e-04
Validation binary_cross_entropy = 1.600365
Epoch 295
Validation binary_cross_entropy = 1.606166
Epoch 296
Validation binary_cross_entropy = 1.612115
Epoch 297
Validation binary_cross_entropy = 1.618303
Epoch 298
Validation binary_cross_entropy = 1.624167
Epoch 299
Loss = 9.1814e-05, PNorm = 95.1990, GNorm = 0.0019, lr_0 = 6.0621e-04
Validation binary_cross_entropy = 1.629682
Model 0 best validation binary_cross_entropy = 0.270928 on epoch 15
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test binary_cross_entropy = 0.234550
Ensemble test binary_cross_entropy = 0.234550
10-fold cross validation
	Seed 0 ==> test binary_cross_entropy = 0.183475
	Seed 1 ==> test binary_cross_entropy = 0.171303
	Seed 2 ==> test binary_cross_entropy = 0.177341
	Seed 3 ==> test binary_cross_entropy = 0.226705
	Seed 4 ==> test binary_cross_entropy = 0.167744
	Seed 5 ==> test binary_cross_entropy = 0.197010
	Seed 6 ==> test binary_cross_entropy = 0.210249
	Seed 7 ==> test binary_cross_entropy = 0.207512
	Seed 8 ==> test binary_cross_entropy = 0.187627
	Seed 9 ==> test binary_cross_entropy = 0.234550
Overall test binary_cross_entropy = 0.196352 +/- 0.021789
Elapsed time = 0:07:30
